---
title: 'Artificial Intelligence: American Attitudes and Trends'
author: "Baobao Zhang and Allan Dafoe"
date: "January 2019"
output:
  bookdown::pdf_document2:
    fig_caption: yes
    includes:
      in_header: texheader.tex
    latex_engine: xelatex
    number_sections: yes
    toc: yes
    toc_depth: 2
editor_options:
  chunk_output_type: console
institute: Center for the Governance of AI, University of Oxford
link-citations: yes
keep_tex: yes
linkcolor: blue
citecolor: blue
subparagraph: yes
bibliography: us_2018_bib.bib
urlcolor: blue
---

\Urlmuskip=0mu plus 1mu\relax

\newpage


```{r global_options, include=FALSE}
  knitr::opts_chunk$set(fig.pos = 'H')
```

\addtocontents{toc}{\protect\setcounter{tocdepth}{0}}
\counterwithin{figure}{section}
\counterwithin{table}{section}

# Acknowledgements {-}

## Primary researchers {-}

**[Baobao Zhang](https://baobaofzhang.github.io/)**

Research Affiliate, Center for the Governance of AI, Future of Humanity Institute, University of Oxford

PhD Candidate, Department of Political Science, Yale University 

**[Allan Dafoe](https://www.allandafoe.com/)**

Director, Center for the Governance of AI, Future of Humanity Institute, University of Oxford

Associate Professor and Senior Research Fellow in the International Politics of AI, University of Oxford

## Editing and design {-}

For useful feedback we would like to thank: Miles Brundage, Jack Clark, Kanta Dihal, Jeffrey Ding, Carrick Flynn, Ben Garfinkel, Rose Hadshar, Tim Hwang, Katelynn Kyker, Jade Leung, Luke Muehlhauser, Cullen O’Keefe, Michael Page, William Rathje, Carl Shulman, Brian Tse, Remco Zwetsloot, and the YouGov Team (Marissa Shih and Sam Luks). In particular, we are grateful for Markus Anderljung's insightful suggestions and detailed editing. 

Copy editor: Steven Van Tassell

Cover design: Laura Pomarius

Web design: Baobao Zhang

Research assistants: Will Marks and Catherine Peng

## Funders {-}

The research was funded by the [Ethics and Governance of Artificial Intelligence Fund](https://aiethicsinitiative.org/) and [Good Ventures](http://www.goodventures.org/).

## For media or other inquiries {-}

Baobao Zhang

+1 (813) 368-9992

Email: surveys@governance.ai

Website: [www.governance.ai](www.governance.ai)

We invite suggestions for questions and partnership opportunities for future survey waves. 

## Recommended citation {-}

Zhang, Baobao and Allan Dafoe. "Artificial Intelligence: American Attitudes and Trends." Oxford, UK: Center for the Governance of AI, Future of Humanity Institute, University of Oxford, 2019. 

\newpage

\addtocontents{toc}{\protect\setcounter{tocdepth}{2}}

# Executive summary 

Advances in artificial intelligence (AI)[^aideffn] could impact nearly all aspects of society: the labor market, transportation, healthcare, education, and national security. AI's effects may be profoundly positive, but the technology entails risks and disruptions that warrant attention. While technologists and policymakers have begun to discuss AI and applications of machine learning more frequently, public opinion has not shaped much of these conversations. In the U.S., public sentiments have shaped many policy debates, including those about immigration, free trade, international conflicts, and climate change mitigation. As in these other policy domains, we expect the public to become more influential over time. It is thus vital to have a better understanding of how the public thinks about AI and the governance of AI. Such understanding is essential to crafting informed policy and identifying opportunities to educate the public about AI's character, benefits, and risks.

In this report, we present the results from an extensive look at the American public’s attitudes toward AI and AI governance. As the study of the public opinion toward AI is relatively new, we aimed for breadth over depth, with our questions touching on: workplace automation; attitudes regarding international cooperation; the public’s trust in various actors to develop and regulate AI; views about the importance and likely impact of different AI governance challenges; and historical and cross-national trends in public opinion regarding AI. Our results provide preliminary insights into the character of U.S. public opinion regarding AI. However, our findings raise more questions than they answer; they are more suggestive than conclusive. Accordingly, **we recommend caution** in interpreting the results; we confine ourselves to primarily reporting the results. More work is needed to gain a deeper understanding of public opinion toward AI. 

Supported by a grant from the Ethics and Governance of AI Fund, we intend to conduct more extensive and intensive surveys in the coming years, including of residents in Europe, China, and other countries. We welcome collaborators, especially experts on particular policy domains, on future surveys. Survey inquiries can be emailed to [surveys@governance.ai](mailto:surveys@governance.ai).   

[^aideffn]: We define AI as machine systems capable of sophisticated (intelligent) information processing. For other definitions, see Footnote 2 in @dafoe2018aigovernance. 

This report is based on findings from a nationally representative survey conducted by the [Center for the Governance of AI](https://www.governance.ai), housed at the Future of Humanity Institute, University of Oxford, using the survey firm YouGov. The survey was conducted between June 6 and 14, 2018, with a total of 2,000 American adults (18+) completing the survey. The analysis of this survey was [pre-registered on the Open Science Framework](https://osf.io/7gqvm/). [Appendix A](#appmethod) provides further details regarding the data collection and analysis process. 

## Select results

Below we highlight some results from our survey[^chronorderfn]: 

[^chronorderfn]: These results are presented roughly in the order in which questions were presented to respondents. 

- Americans express [mixed support for the development of AI](#subsecsupportai). After reading a short explanation, a substantial minority (41%) somewhat support or strongly support the development of AI, while a smaller minority (22%) somewhat or strongly opposes it.  

- Demographic characteristics account for [substantial variation in support for developing AI](#subsecdemosupportai). Substantially more support for developing AI is expressed by college graduates (57%) than those with high school or less education (29%); by those with larger reported household incomes, such as those earning over \$100,000 annually (59%), than those earning less than \$30,000 (33%); by those with computer science or programming experience (58%) than those without (31%); by men (47%) than women (35%). These differences are not easily explained away by other characteristics (they are robust to our multiple regression).

- The [overwhelming majority of Americans (82%)](#subsecsupportmanageai) believe that robots and/or AI should be carefully managed. This figure is comparable to with survey results from EU respondents. 

- Americans consider all of the [thirteen AI governance challenges](#subsecgovchallenges13) presented in the survey to be important for governments and technology companies to manage carefully. The governance challenges perceived to be the most likely to impact people around the world within the next decade _and_ rated the highest in issue importance were[^weightingfn]:

    1. Preventing AI-assisted surveillance from violating privacy and civil liberties
    2. Preventing AI from being used to spread fake and harmful content online
    2. Preventing AI cyber attacks against governments, companies, organizations, and individuals
    4. Protecting data privacy


- We also asked the above question, but focused on the likelihood of the governance challenge impacting solely Americans (rather than people around the world). Americans perceive that all of the governance challenges presented, except for protecting data privacy and ensuring that autonomous vehicles are safe, are [slightly more likely to impact people around the world](#appgovchallenges) than to impact Americans within the next 10 years. 

[^weightingfn]: Giving equal weight to the likelihood and the rated importance of the challenge.

- Americans have discernibly different levels of trust in various organizations to develop and manage[^detailfn] AI for the best interests of the public. Broadly, the public puts the most trust in university researchers (50% reporting "a fair amount of confidence" or "a great deal of confidence") and the U.S. military (49%); followed by scientific organizations, the Partnership on AI, technology companies (excluding Facebook), and intelligence organizations; followed by U.S. federal or state governments, and the UN; followed by Facebook. 

[^detailfn]: Our survey asked separately about trust in 1) building and 2) managing the development and use of AI. Results are similar and are combined here. 

- Americans express mixed support (1) for the U.S. investing more in AI military capabilities _and_ (2) for cooperating with China to avoid the dangers of an AI arms race. [Providing respondents with information about the risks of a U.S.-China AI arms race](#subsecexperimentchina) slightly decreases support for the U.S. investing more in AI military capabilities. Providing a pro-nationalist message or a message about AI's threat to humanity failed to affect Americans' policy preferences.

- The median respondent predicts that there is a [54% chance that high-level machine intelligence will be developed by 2028](#arrivesooner). We define _high-level machine intelligence_ as when machines are able to perform almost all tasks that are economically relevant today better than the median human (today) at each task. See [Appendix B](#forecasthlmi) for a detailed definition. 
 
- Americans express [weak support for developing high-level machine intelligence](#subsecsupporthlmi): 31% of Americans support while 27% oppose its development. 

- Demographic characteristics account for [substantial variation in support for developing high-level machine intelligence](#subsecdemohlmi). There is substantially more support for developing high-level machine intelligence by those with larger reported household incomes, such as those earning over \$100,000 annually (47%) than those earning less than \$30,000 (24%); by those with computer science or programming experience (45%) than those without (23%); by men (39%) than women (25%). These differences are not easily explained away by other characteristics (they are robust to our multiple regression). 

- There are more Americans who think that high-level machine intelligence [will be harmful than those who think it will be beneficial to humanity](#subsecharmgood). While 22% think that the technology will be "on balance bad," 12% think that it would be "extremely bad," leading to possible human extinction. Still, 21% think it will be "on balance good," and 5% think it will be "extremely good." 

## Reading notes

- In all tables and charts, results are weighted to be representative of the U.S. adult population, unless otherwise specified. We use the weights provided by YouGov. 

- Wherever possible, we report the margins of error (MOEs), confidence regions, and error bars at the 95% confidence level.

- For tabulation purposes, percentage points are rounded off to the nearest whole number in the figures. As a result, the percentages in a given figure may total slightly higher or lower than 100%. Summary statistics that include two decimal places are reported in [Appendix B](#apptopline). 

```{r setup, include=FALSE, cache=TRUE, warning=FALSE}
rm(list = ls(all = TRUE))

library(lubridate)
library(wCorr)
library(haven)
library(estimatr)
library(estimatr)
library(Hmisc)
library(magrittr)
library(dplyr)
library(ggplot2)
library(car)
library(sandwich)
library(lmtest)
library(labelled)
library(stringr)
library(scales)
library(pander)
library(knitr)
library(tidyr)
library(ggrepel)
library(questionr)
library(miceadds)
library(kableExtra)
library(shadowtext)
library(clubSandwich)
library(extrafont)
library(ggrepel)
library(broom)
library(countrycode)
loadfonts(device = "pdf")

# Set seed
set.seed(7592)

# Wald Test function

waldtest <- function(object, ...) {
  UseMethod("waldtest")
}

waldtest.formula <- function(object, ..., data = list()) {
  object <- if(length(data) < 1) eval(call("lm", formula = as.formula(deparse(substitute(object))),
    environment(object)))
  else eval(call("lm", formula = as.formula(deparse(substitute(object))),
    data = as.name(deparse(substitute(data))), environment(data)))
  waldtest.lm(object, ...)
}

waldtest.default <- function(object, ..., vcov = NULL, test = c("Chisq", "F"), name = NULL)
{
  ## methods needed:
  ## - terms()
  ## - update()
  ## - formula()
  ## - nobs() or residuals() -> only for determining number of observations
  ## - df.residual() or logLik
  ## - coef() -> needs to be named, matching names in terms() and vcov()
  ## - vcov(), potentially user-supplied

  ## use S4 methods if loaded
  coef0   <- function(x, ...) {
    coef1 <- if("stats4" %in% loadedNamespaces()) stats4::coef else coef
    na.omit(coef1(x, ...))
  }
  logLik0 <- if("stats4" %in% loadedNamespaces()) stats4::logLik else logLik
  update0 <- if("stats4" %in% loadedNamespaces()) stats4::update else update
  nobs0   <- function(x, ...) {
    nobs1 <- if("stats4" %in% loadedNamespaces()) stats4::nobs else nobs
    nobs2 <- function(x, ...) NROW(residuals(x, ...))
    rval <- try(nobs1(x, ...), silent = TRUE)
    if(inherits(rval, "try-error") | is.null(rval)) rval <- nobs2(x, ...)
    return(rval)
  }
  vcov0   <- if(!is.null(vcov)) vcov else {
    if("stats4" %in% loadedNamespaces()) stats4::vcov else stats::vcov
  }
  df.residual0 <- function(x) {
    df <- try(df.residual(x), silent = TRUE)
    if(inherits(df, "try-error") | is.null(df)) df <- try(nobs0(x) - attr(logLik0(x), "df"), silent = TRUE)
    if(inherits(df, "try-error") | is.null(df)) df <- try(nobs0(x) - length(as.vector(coef0(x))), silent = TRUE)
    if(inherits(df, "try-error")) df <- NULL
    return(df)
  }

  ## model class
  cls <- class(object)[1]

  ## convenience functions:
  ## 1. extracts term labels
  tlab <- function(x) {
    tt <- try(terms(x), silent = TRUE)
    if(inherits(tt, "try-error")) "" else attr(tt, "term.labels")
  }
  ## 2. extracts model name
  if(is.null(name)) name <- function(x) {
    rval <- try(formula(x), silent = TRUE)
    if(inherits(rval, "try-error") | is.null(rval)) rval <- try(x$call, silent = TRUE)
    if(inherits(rval, "try-error") | is.null(rval)) return(NULL) else return(paste(deparse(rval), collapse="\n"))
  }
  ## 3. compute an updated model object
  modelUpdate <- function(fm, update) {
    ## if `update' is numeric or character, then assume that the 
    ## corresponding variables (i.e., terms) are redundant (i.e., should be omitted)
    if(is.numeric(update)) {
      ## sanity checking of numeric update specification
      if(any(update < 1)) {
        warning("for numeric model specifications all values have to be >=1")
	update <- abs(update)[abs(update) > 0]
      }
      if(any(update > length(tlab(fm)))) {
        warning(paste("more terms specified than existent in the model:",
	        paste(as.character(update[update > length(tlab(fm))]), collapse = ", ")))
	update <- update[update <= length(tlab(fm))]
      }
      ## finally turn numeric into character update specification
      update <- tlab(fm)[update]
    }
    if(is.character(update)) {
      ## sanity checking of character update specification
      if(!all(update %in% tlab(fm))) {
        warning(paste("terms specified that are not in the model:",
	        paste(dQuote(update[!(update %in% tlab(fm))]), collapse = ", ")))
        update <- update[update %in% tlab(fm)]
      }
      if(length(update) < 1) stop("empty model specification")  
      ## finally turn character into formula update specification       
      update <- as.formula(paste(". ~ . -", paste(update, collapse = " - ")))
    }
    if(inherits(update, "formula")) update <- update0(fm, update)
    if(!inherits(update, cls)) stop(paste("original model was of class \"", cls,
      "\", updated model is of class \"", class(update)[1], "\"", sep = ""))
    return(update)
  }
  ## 4. compare two fitted model objects
  modelCompare <- function(fm, fm.up, vfun = NULL) {
    q <- length(coef0(fm)) - length(coef0(fm.up))

    if(q > 0) {
      fm0 <- fm.up
      fm1 <- fm
    } else {
      fm0 <- fm
      fm1 <- fm.up
    }
    k <- length(coef0(fm1))
    n <- nobs0(fm1)

    ## determine omitted variables
    if(!all(tlab(fm0) %in% tlab(fm1))) stop("models are not nested")
    ovar <- which(!(names(coef0(fm1)) %in% names(coef0(fm0))))

    ## get covariance matrix estimate
    vc <- if(is.null(vfun)) vcov(fm1)
          else if(is.function(vfun)) vfun(fm1)
	  else vfun

    ## compute Chisq statistic
    stat <- t(coef0(fm1)[ovar]) %*% solve(vc[ovar,ovar], tol = 1e-100) %*% coef0(fm1)[ovar]
    return(c(-q, stat))
  }

  ## recursively fit all objects (if necessary)
  objects <- list(object, ...)
  nmodels <- length(objects)
  if(nmodels < 2) {
    objects <- c(objects, . ~ 1)
    nmodels <- 2
  }
  
  # remember which models are already fitted and which are described
  # by an update mechanism
  no.update <- sapply(objects, function(obj) inherits(obj, cls))
  
  ## updating
  for(i in 2:nmodels) objects[[i]] <- modelUpdate(objects[[i-1]], objects[[i]])

  ## check responses
  getresponse <- function(x) {
    tt <- try(terms(x), silent = TRUE)
    if(inherits(tt, "try-error")) "" else deparse(tt[[2]])
  }
  responses <- as.character(lapply(objects, getresponse))
  sameresp <- responses == responses[1]
  if(!all(sameresp)) {
    objects <- objects[sameresp]
    warning("models with response ", deparse(responses[!sameresp]),
	    " removed because response differs from ", "model 1")
  }

  ## check sample sizes
  ns <- sapply(objects, nobs0)
  if(any(ns != ns[1])) {
    for(i in 2:nmodels) {
      if(ns[1] != ns[i]) {
        if(no.update[i]) stop("models were not all fitted to the same size of dataset")
	  else {
	    commonobs <- row.names(model.frame(objects[[i]])) %in% row.names(model.frame(objects[[i-1]]))
	    objects[[i]] <- eval(substitute(update(objects[[i]], subset = commonobs),
	      list(commonobs = commonobs)))
	    if(nobs0(objects[[i]]) != ns[1]) stop("models could not be fitted to the same size of dataset")
	  }
      }
    }
  }

  ## check vcov0
  if(nmodels > 2 && !is.null(vcov0) && !is.function(vcov0))
    stop("to compare more than 2 models `vcov' needs to be a function")

  ## setup ANOVA matrix
  test <- match.arg(test)
  rval <- matrix(rep(NA, 4 * nmodels), ncol = 4)
  colnames(rval) <- c("Res.Df", "Df", test, paste("Pr(>", test, ")", sep = ""))
  rownames(rval) <- 1:nmodels
  rval[,1] <- as.numeric(sapply(objects, df.residual0))
  for(i in 2:nmodels) rval[i, 2:3] <- modelCompare(objects[[i-1]], objects[[i]], vfun = vcov0)
  if(test == "Chisq") {
    rval[,4] <- pchisq(rval[,3], round(abs(rval[,2])), lower.tail = FALSE)
  } else {
    df <- rval[,1]
    for(i in 2:nmodels) if(rval[i,2] < 0) df[i] <- rval[i-1,1]
    rval[,3] <- rval[,3]/abs(rval[,2])
    rval[,4] <- pf(rval[,3], abs(rval[,2]), df, lower.tail = FALSE)
  }

  variables <- lapply(objects, name)
  if(any(sapply(variables, is.null))) variables <- lapply(match.call()[-1L], deparse)[1L:nmodels]
  title <- "Wald test\n"
  topnote <- paste("Model ", format(1:nmodels),": ", variables, sep="", collapse="\n")

  structure(as.data.frame(rval), heading = c(title, topnote),
	    class = c("anova", "data.frame"))
}

waldtest.lm <- function(object, ..., test = c("F", "Chisq"))
{
  if(!is.null(alias(object)$Complete)) stop("there are aliased coefficients in the model")
  waldtest.default(object, ..., test = match.arg(test))
}


# Text wrapper function
wrapper <- function(x, ...) {
  paste(strwrap(x, ...), collapse = "\n")
}

# Function to create bivariate mean confidence region
bivCI <- function(s, xbar, n, alpha, m, population = FALSE) {
  x <- sin(2 * pi * (0:(m - 1)) / (m - 1))
  y <- cos(2 * pi * (0:(m - 1)) / (m - 1))
  cv <- qchisq(1 - alpha, 2)
  cv <- if (population) {
    cv
  } else {
    cv / n
  }
  for (i in 1:m) {
    pair <- c(x[i], y[i])
    q <- pair %*% solve(s, pair)
    x[i] <- x[i] * sqrt(cv / q) + xbar[1]
    y[i] <- y[i] * sqrt(cv / q) + xbar[2]
  }
  data.frame(x, y)
}

# Trailing zeros rounding function
roundfunc <- function(x,
                      round_digits = 2,
                      lessthan = TRUE) {
  if (lessthan) {
    temp <- ifelse(x > 0 & round(x, round_digits) == 0,
                   paste0("<0.", rep(0, (round_digits - 1)), 1),
                   sprintf(paste0("%.", round_digits, "f"), round(x, round_digits)))
    temp <- ifelse(x < 0 & round(x, round_digits) == 0,
                   paste0(">-0.", rep(0, (round_digits - 1)), 1),
                   temp)
    temp[x == 0] <- 0
    return(temp)
  } else {
    return(sprintf(paste0("%.", round_digits, "f"), round(x, round_digits)))
  }
}

roundfunc(x = 0.001, round_digits = 2)
relabel_var <- function(old_var, old_labels, new_labels) {
  new_var <- rep(NA, length(old_var))
  if (is.factor(old_var)) {
    old_var <- as.character(old_var)
  }
  for (i in 1:length(old_labels)) {
    new_var[old_var == old_labels[i]] <- new_labels[i]
  }
  return(new_var)
}

relabel_var2 <- function(old_var, old_labels, new_labels) {
  new_var <- rep(NA, length(old_var))
  if (is.factor(old_var)) {
    old_var <- as.character(old_var)
  }
  for (i in 1:length(old_labels)) {
    new_var[old_var %in% old_labels[[i]]] <- new_labels[i]
  }
  return(new_var)
}

catvar_func <-
  function(outcome,
           outcome_var,
           label_var,
           num_missing,
           num_DK,
           shown,
           output_type,
           new_values,
           edit_labels = TRUE,
           survey_weights = d$survey_weights, id = d$caseid,
           missing_recode) {
    # Clean data to make the bar chart
    # Get the value labels
    value_labels <- as.data.frame(labelled::val_labels(label_var))
    value_labels$labels <- row.names(value_labels)
    names(value_labels)[1] <- "num"
    row.names(value_labels) <- NULL
    # Make the frequency table
    sum_func <- function(outcome_var, value, survey_weights) {
      se_md <- lm(outcome_var[shown] == value ~ 1, 
                  weights = survey_weights[shown])
      out <- coeftest(se_md, vcov = vcovHC(se_md, type="HC2"))
      return(data.frame(num = value, 
        Freq = sum(outcome_var[shown] == value),
                        Prop = as.numeric(out[1]), 
                        se = as.numeric(out[2])))
    }
    value_table <- do.call(rbind, lapply(value_labels$num, sum_func,
                          outcome_var = outcome_var,
           survey_weights = survey_weights))
    # Merge the frequency table with the value labels
    value_table <-
      merge(x = value_table, y = value_labels, all.y = TRUE)
    value_table$group <-
      ifelse(value_table$num %in% c(num_missing, num_DK),
             "Don't know/Missing",
             "Responses")
    value_table$group <-
      factor(value_table$group,
             levels = c("Responses",
                        "Don't know/Missing"))
    value_table$labels <- Hmisc::capitalize(value_table$labels)
    value_table$outcome <- outcome
    value_table$new_values <- new_values
    if (edit_labels) {
      value_table$labels <- ifelse(
        value_table$group == "Responses",
        paste0(value_table$new_values, ". ",
               value_table$labels),
        value_table$labels
      )
    }
    # Remove the not asked
    value_table <- value_table[!grepl(pattern = "not asked", 
                                      value_table$labels, 
                                      ignore.case = TRUE),]
    # Get the summary statistics
    num_outcome <- as.numeric(outcome_var[shown])
    survey_weights <- survey_weights[shown]
    num_outcome <-
      relabel_var(
        old_var = num_outcome,
        old_labels = value_table$num,
        new_labels = value_table$new_values
      )
    num_outcome_missing <- is.na(num_outcome)
    num_outcome[is.na(num_outcome)] <- missing_recode
    # Get the percent missing
    percent_missing <-
      sum(num_outcome_missing) / length(num_outcome_missing)
    # Get the mean and se
    md <- if (percent_missing > 0.1) {
      # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      se_md <- lm(num_outcome ~ scale(num_outcome_missing),
                 weights = survey_weights)
      coeftest(se_md, vcov = vcovHC(se_md, type="HC2"))[1,]
    } else {
      se_md <- lm(num_outcome ~ 1,
                 weights = survey_weights)
      coeftest(se_md, vcov = vcovHC(se_md, type="HC2"))
    }
    # Put the summary statistics together
    value_sum <-
      data.frame(
        outcome = outcome,
        num = md[1],
        se = md[2],
        group = "Responses",
        sum_stat = paste0("Mean: ", roundfunc(md[1]), 
                          " (MOE: +/-", roundfunc(qnorm(0.975)*md[2]),
                          "); N = ",
                          sum(shown)),
        N = sum(shown)
      )
    if (output_type == "num_outcome") {
      return(num_outcome)
    } else if (output_type == "value_table") {
      return(value_table)
    } else {
      return(value_sum)
    }
  }

# CDF fitting functions
gamma.dist.f <-
  function(x, shape, scale) {
    pgamma(x, shape = shape, scale = scale)
  }


fit <- function(x, ps, distribution, par_init) {
  err = function (data, par) {
    curve = distribution(data$x, par[1], par[2])
    sum((curve - data$y) ^ 2)
  }
  
  data = data.frame(x = x, y = ps)
  
  i = data.frame(pars = I(par_init)) %>%
    rowwise %>%
    mutate(error = err(data, pars)) %>%
    slice(which.min(error)) %>% .$pars
  
  initial_params = i[[1]]
  
  
  
  result = suppressWarnings(optim(par = initial_params,
                                  fn = err,
                                  data = data))
  data.frame(
    shape = result$par[1],
    scale = result$par[2],
    convergence = result$convergence,
    error = result$value
  )
}


fit.all <- function (data, distribution, par_init) {
  data %>%
    group_by(response.id) %>%
    do(fit(.$x, .$p, distribution, par_init = par_init)) %>%
    ungroup()
  
}

cum.dist <- function(params, x, distribution) {
  merge(x, params) %>%
    mutate(p = distribution(x, shape, scale))
}

gen.cum.dists <- function(df, x, dist, init) {
  df %>%
    fit.all(dist, par_init = init) %>%
    cum.dist(x, dist)
}

# Weighted summary statistics function
md_weight <- function(x, weights, which_stat) {
  lm_md <- lm(x ~ 1, weights = weights)
  md <- coeftest(lm_md, vcov = vcovHC(lm_md, type="HC2"))[,1:4]
  if (which_stat == "mean") {
    return(md[1])
  } else {
    return(md[2])
  }
}

# Regression analysis
regression_output <- function(formula, variable_names,
         caption, output_model = FALSE, survey_weights,
         kable_output = NULL, shown = rep(TRUE, nrow(d))) {
  md <- lm(
    as.formula(formula),
    data = d, subset = shown, weights = survey_weights
  )
  vcovHC_res <- vcovHC(md, type="HC2")
  reg_table <- coeftest(md, vcov = vcovHC_res)
  reg_f <- waldtest(md, vcov = vcovHC_res)
  reg_stars <- rep("", nrow(reg_table))
  reg_stars[reg_table[, 4] < 0.05] <- "*"
  reg_stars[reg_table[, 4] < 0.01] <- "**"
  reg_stars[reg_table[, 4] < 0.001] <- "***"
  coef <-
    paste0(roundfunc(reg_table[, 1]),
           " (",
           roundfunc(reg_table[, 2]),
           ")",
           reg_stars)
  f_stat_p <- reg_f$`Pr(>F)`[2]
  f_stat <-
    paste0(
      "\\textit{F}(",
      -1*reg_f$Df[2],
      ", ",
      reg_f$Res.Df[1],
      ") = ",
      roundfunc(reg_f$F[2]),
      "; \\textit{p}-value: ",
      ifelse(f_stat_p < 0.001, "<0.001", round(f_stat_p, 3))
    )
  # Generate the regression table
  if (output_model) {
    return(md)
  } else {
    output_sum <- data.frame(vars = c(variable_names, paste0("\\textit{N} = ", nrow(d[shown,]))),
                             coef = c(coef, f_stat))
    if (is.null(kable_output)) {
      return(kable(x = output_sum, format = "pandoc",
                   caption = caption, col.names = c("Variables", "Coefficients (SEs)")))
    } else {
      return(kable(x = output_sum[kable_output,],
                   caption = caption, format = "pandoc",
                   col.names = c("Variables", "Coefficients (SEs)"), row.names = FALSE))
    }
  }
}


# Set working directory
wd <- "~/Google Drive/AI Public Opinion Surveys/R/dataverse_jan2019_report/"

# Load datasets
d0 <- read_sav(paste0(wd, "YALE0065_OUTPUT.sav"))

# Sample with replacement to create 2000 responses
d <- as.data.frame(d0)
d$r_id <- 1:nrow(d)
# Survey weights
d$survey_weights <- d$weight
# Equal weights
d$weight1 <- 1

# Median values for the answer choice ranges
mc_p_med <- c(median(0:5), median(5:20), median(20:40), median(40:60),
              median(60:80), median(80:95), median(95:100))


# Merge in the time variable
d_time <- read.csv(paste0(wd, "YALE0065_field-date.csv"),
                   stringsAsFactors=FALSE)
d_time$starttime <- lubridate::ymd_hms(d_time$starttime)
d_time$endtime <- lubridate::ymd_hms(d_time$endtime)
d_time$surveytime <- d_time$endtime - d_time$starttime
d$starttime <- NULL # remove the old variables
d$endtime <- NULL
# Merge the two datasets
d <- merge(x = d, y = d_time, all.x = TRUE, by = "caseid")


# Create some variables: number of non-selected
d$whatsai_s <- rowSums(d[,paste0("Q3new_", 1:10)] == 1)
d$whatsrobot_s <- rowSums(d[,paste0("Q3new_", c(8, 9))] == 1)

# Create the spend spent variables
d$surveytime <- as.numeric(d$surveytime, units = "mins")
d$surveytime_dev <- 
  abs(d$surveytime - median(d$surveytime))

# Label the experimental groups
d$q3new_treat_c <- relabel_var(old_var = d$q3new_treat, old_labels = 1:4, 
            new_labels = c("AI", "Automation", "Machine learning", "Robotics"))

```

\newpage

# General attitudes toward AI

\FloatBarrier

## More Americans support than oppose developing AI {#subsecsupportai}

We measured respondents' support for the further development of AI after providing them with basic information about the technology. Respondents were given the following definition of AI:

> Artificial Intelligence (AI) refers to computer systems that perform tasks or make decisions that usually require human intelligence. AI can perform these tasks or make these decisions without explicit human instructions. Today, AI has been used in the following applications: [five randomly selected applications]

Each respondent viewed five applications randomly selected from a list of 14 that included translation, image classification, and disease diagnosis. Afterward, respondents were asked how much they support or oppose the development of AI. (See [Appendix B](#supportdevai) for the list of the 14 applications and the survey question.)

```{r supportdevrisks, echo=FALSE, fig.height=4, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.cap="Support for developing AI"}

dev_ai_overall_mean <- wtd.mean(relabel_var(d$Q5, c(1:6, 8, 9), c(2:-2, NA, NA, NA)),
                            d$survey_weights, na.rm = TRUE)
# Frequency table
dev_ai_value_table <- catvar_func(
  outcome = label(d0$Q5),
  outcome_var = d$Q5,
  label_var = d0$Q5,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6,
  new_values <- c(2:-2, NA, NA, NA),
  survey_weights = d$survey_weights, 
  missing_recode = dev_ai_overall_mean
  )  
dev_ai_value_table$num <- c(2, 1, 0, -1, -2, 8, 9)
# Numerical values
dev_ai_value_sum <- catvar_func(
  outcome = label(d0$Q5),
  outcome_var = d$Q5,
  label_var = d0$Q5,
  output_type = "value_sum",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6,
  new_values <- c(2, 1, 0, -1, -2, NA, NA, NA),
  survey_weights = d$survey_weights,
  missing_recode = dev_ai_overall_mean
  )  


dev_ai_value_table$labels <- factor(dev_ai_value_table$labels, 
                                    levels = dev_ai_value_table$labels[c(5:1, 6:7)])

# Make the chart
ggplot() +
  geom_bar(data = dev_ai_value_table[dev_ai_value_table$Prop !=0,], 
           aes(x = num, y = Prop, fill = labels), 
           stat = "identity", alpha = 0.6, color = "black") +
  geom_errorbar(data = 
                  dev_ai_value_table[dev_ai_value_table$Prop !=0,], 
                aes(x = num, ymin = Prop + qnorm(0.025)*se,
                    ymax = Prop + qnorm(0.975)*se), width = 0.1) +
  geom_text(data = dev_ai_value_table, aes(x = num, 
                                           label = roundfunc(Prop*100, 0)), 
            y = 0.02, nudge_x = 0.25) +
  scale_x_continuous(breaks = dev_ai_value_table$num[order(dev_ai_value_table$num)],
    labels = str_wrap(dev_ai_value_table$labels[order(dev_ai_value_table$num)], 
                      width = 15)) +
  facet_grid(~group, scales = "free_x", space = "free_x") + theme_bw() +
  geom_text(data = dev_ai_value_sum, aes(x = 0, label = sum_stat,
                                       y = max(dev_ai_value_table$Prop)+0.05)) +
  scale_fill_manual(values = c("#ca0020", "#f4a582", "#f7f7f7", "#92c5de", "#0571b0",
                               "grey65", "grey65")) + 
  scale_y_continuous(labels = scales::percent, 
                     limits = c(0, max(dev_ai_value_table$Prop)+0.05)) +
  xlab("Responses") + ylab("Percentage of respondents") + theme(legend.position="none") +
  labs(caption = "Source: Center for the Governance of AI")

```

Americans express mixed support for the development of AI, although more support than oppose the development of AI, as shown in Figure \@ref(fig:supportdevrisks). A substantial minority (41%) somewhat or strongly supports the development of AI. A smaller minority (22%) somewhat or strongly oppose its development. Many express a neutral attitude: 28% of respondents state that they neither support nor oppose while 10% indicate they do not know.

Our survey results reflect the cautious optimism that Americans express in other polls. In a recent survey, 51% of Americans indicated that they support continuing AI research while 31% opposed it [@morningconsult2017]. Furthermore, 77% of Americans expressed that AI would have a "very positive" or "mostly positive" impact on how people work and live in the next 10 years, while 23% thought that AI's impact would be "very negative" or "mostly negative" [@negallup2018]. 

## Support for developing AI is greater among those who are wealthy, educated, male, or have experience with technology {#subsecdemosupportai}

We examined support for developing AI by 11 demographic subgroup variables, including age, gender, race, and education. (See [Appendix A](#appdemosubgroups) for descriptions of the demographic subgroups.) We performed a multiple linear regression to predict support for developing AI using all these demographic variables. 

```{r demographicsupportstackbar, echo=FALSE, fig.height=10, fig.keep='all', cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf', fig.width=8.5, fig.cap="Support for developing AI across demographic characteristics: distribution of responses"}

# Age
age_levels <- c("Age 18-37", "Age 38-53", "Age 54-72", "Age 73 and older")
d$demo_age <- relabel_var2(old_var = d$birthyr, old_labels = list(1981:2018, 1965:1980,
                                                     1946:1964, 1900:1945),
                           new_labels = age_levels)
d$demo_age <- factor(d$demo_age, levels = age_levels)
# Gender
d$demo_gender <- ifelse(d$gender == 1, "Male", "Female")
d$demo_gender <- factor(d$demo_gender, c("Female", "Male"))
# Race
d$demo_white <- ifelse(d$race == 1, "White", "Non-white")
d$demo_white <- factor(d$demo_white, c("White", "Non-white"))
# Education
d$demo_educ <- relabel_var2(old_var = d$educ, 
                              old_labels = list(1:2, 3:4, 5:6),
                           new_labels = c("HS or less", "Some college", "College+"))
d$demo_educ <- factor(d$demo_educ, 
                      levels = c("HS or less", "Some college", "College+"))
# Employed
d$demo_employ <- ifelse(d$employ %in% c(1, 2), "Employed (full- or part-time)", 
                        "Not employed")
d$demo_employ <- factor(d$demo_employ, levels = c("Not employed", 
                                                  "Employed (full- or part-time)"))
# Income
d$demo_income <- relabel_var2(old_var = d$faminc_new, 
                              old_labels = list(1:3, 4:7, 8:9, 10:16, 97),
                           new_labels = c("Income less than $30K", 
                                          "Income $30-70K", "Income $70-100K",
                                          "Income more than $100K", 
                                          "Prefer not to say income"))
d$demo_income <- factor(d$demo_income, levels = c("Income less than $30K", 
                                          "Income $30-70K", "Income $70-100K",
                                          "Income more than $100K", 
                                          "Prefer not to say income"))
# Political 
d$demo_pid3 <- relabel_var2(old_var = d$pid3, old_labels = list(1, 2, 3:5),
                           new_labels = c("Democrat", "Republican", "Independent/Other"))
d$demo_pid3 <- factor(d$demo_pid3, levels = c("Republican", "Democrat", "Independent/Other"))
d$demo_rel <- ifelse(d$religpew %in% 1:4, "Christian", "Other religion")
d$demo_rel  <- factor(d$demo_rel, levels = c("Christian", "No religious affiliation",
                                             "Other religion"))
d$demo_rel[d$religpew %in% c(9, 10, 11)] <- "No religious affiliation"
d$demo_bornagain <- ifelse(d$pew_bornagain == 1, 
                           "Born-again Christian", "Not born-again Christian")
d$demo_bornagain <- factor(d$demo_bornagain, levels = c("Not born-again Christian", "Born-again Christian"))
d$demo_cs <- ifelse(d$Q4_2 == 1 | d$Q4_3 == 1,
                    "CS or engineering degree", "No CS or engineering degree")
d$demo_cs <- factor(d$demo_cs, levels = c("No CS or engineering degree", 
                                          "CS or engineering degree"))
d$demo_prog <- ifelse(d$Q4_1 == 1 | d$Q4_2 == 1 | d$Q4_3 == 1 | d$Q4_4 == 1,
                    "CS or programming experience", "No CS or programming experience")
d$demo_prog <- factor(d$demo_prog, levels = c("No CS or programming experience", "CS or programming experience"))
# Names of the demographic variables
demo_var <- names(d)[grep(pattern = "demo_", ignore.case = FALSE, x = names(d))]
# Clean up the data
d$Q5_clean <- relabel_var(old_var = d$Q5, old_labels = c(1:6, 8, 9),
                          new_labels = c(2, 1, 0, -1, -2, NA, NA, NA))
d$Q5_clean[is.na(d$Q5_clean)] <- dev_ai_overall_mean
d$Q5_clean <- scale(d$Q5_clean)

# Helper function 
demo_support <- function(demo, demo_value, outcome_var = d$Q5, label_var = d0$Q5,
                         demo_group, output_type = "value_sum") {
  temp <- catvar_func(
  outcome = label(label_var),
  outcome_var = outcome_var,
  label_var = label_var,
  output_type = output_type,
  shown = d[,demo] == demo_value,
  num_missing = 8,
  num_DK = 6,
  new_values <- c(2, 1, 0, -1, -2, NA, NA, NA),
  survey_weights = d$survey_weights,
  missing_recode = dev_ai_overall_mean
  )    
  return(data.frame(demo, demo_value, demo_group, temp))
}
# Function to aggregate
demo_support_values <- function(demo, demo_group, output_type = "value_sum") {
  levels_demo <- levels(d[,demo])
  lapply(levels_demo, demo_support, demo = demo, 
         demo_group = demo_group, output_type = output_type) %>% do.call(what = rbind)
}

# Make the summary statistics
demo_group <- c("Age group", "Gender", "Race", "Education", "Employment status",
                "Political party", "Religion", "Born-again Christian",
                "CS or engineering degree", "CS or programming experience")
d_value_sum_support <- rbind(
  demo_support_values(demo = "demo_age", demo_group = "Age group"),
  demo_support_values(demo = "demo_gender", demo_group = "Gender"),
  demo_support_values(demo = "demo_white", demo_group = "Race"),
  demo_support_values(demo = "demo_educ", demo_group = "Education"),
  demo_support_values(demo = "demo_employ", demo_group = "Employment status"),
  demo_support_values(demo = "demo_income", demo_group = "Income"),
  demo_support_values(demo = "demo_pid3", demo_group = "Political party"),
  demo_support_values(demo = "demo_rel", demo_group = "Religion"),
  demo_support_values(demo = "demo_bornagain", demo_group = "Born-again Christian"),
  demo_support_values(demo = "demo_cs", demo_group = "CS or engineering degree"),
  demo_support_values(demo = "demo_prog", demo_group = "CS or programming experience"))

# Make the value frequency table
d_value_table_support <- rbind(
  demo_support_values(demo = "demo_age", demo_group = "Age group",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_gender", demo_group = "Gender",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_white", demo_group = "Race",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_educ", demo_group = "Education",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_employ", demo_group = "Employment status",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_income", demo_group = "Income",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_pid3", demo_group = "Political party",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_rel", demo_group = "Religion",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_bornagain", demo_group = "Born-again Christian",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_cs", demo_group = "CS or engineering degree",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_prog", demo_group = "CS or programming experience",
                      output_type = "value_table"))

# Make stacked bar chart
d_value_table_support$labels[d_value_table_support$labels %in%
                               c("I don't know", "Skipped")] <- 
  "Don't know/Skipped"
# Clean up the data
d_value_table_support <- 
  d_value_table_support %>% group_by(demo_value, demo_group, labels) %>%
  dplyr::summarise(Prop = sum(Prop))

# Make the graph
# Change the factor levels
d_value_table_support$demo_value <- factor(d_value_table_support$demo_value,
                                           levels = rev(levels(d_value_table_support$demo_value)))
# Change the factors
d_value_table_support$labels <- factor(d_value_table_support$labels,
     levels = rev(c("2. Strongly support", 
                "1. Somewhat support",
                "0. Neither support nor oppose", "-1. Somewhat oppose",
                "-2. Strongly oppose", "Don't know/Skipped")))
d_value_table_support$percent <- 
  roundfunc(d_value_table_support$Prop*100, 0)

# Make the graph
ggplot(data = d_value_table_support,
       aes(x=demo_value, y=Prop, fill=labels)) +
  geom_bar(stat="identity", position = "fill", alpha = 0.6) +
  geom_text(aes(label = percent),
            position = position_stack(vjust = 0.5), size = 3) +
  xlab("Demographic subgroups") +
  scale_y_continuous(name = "Percentage of respondents", 
                     labels = scales::percent,
                     limits = c(0, 1), expand = c(0, 0)) +
  coord_flip() +
  theme_bw() +
  facet_grid(demo_group~., scales = "free_y", space = "free_y") +
  scale_fill_manual(values = c("grey65", "#ca0020", "#f4a582", "#f7f7f7", "#92c5de", "#0571b0"), name = "Responses") +
   guides(fill = guide_legend(reverse = TRUE, ncol = 2)) +
  labs(caption = "Source: Center for the Governance of AI") +
  theme(strip.background = element_blank(),
   strip.text.y = element_blank(),
                     legend.position = "bottom",
   axis.text.x = element_text(hjust=1))

```

Support for developing AI varies greatly between demographic subgroups, with gender, education, income, and experience being key predictors. As seen in Figure \@ref(fig:demographicsupportstackbar), a majority of respondents in each of the following four subgroups express support for developing AI: those with four-year college degrees (57%), those with an annual household income above \$100,000 (59%), those who have completed a computer science or engineering degree (56%), and those with computer science or programming experience (58%). In contrast, women (35%), those with a high school degree or less (29%), and those with an annual household income below \$30,000 (33%), are much less enthusiastic about developing AI. One possible explanation for these results is that subgroups that are more vulnerable to workplace automation express less enthusiasm for developing AI. Within developed countries, women, those with low levels of education, and low-income workers have jobs that are at higher risk of automation, according to an analysis by the Organisation for Economic Co-operation and Development [@nedelkoska2018automation].

We used a multiple regression that includes all of the demographic variables to predict support for developing AI. The support for developing AI outcome variable was standardized, such that it has mean 0 and unit variance. 

Significant predictors of _support_ for developing AI include:

- Being a Millennial/post-Millennial (versus being a Gen Xer or Baby Boomer)
- Being a male (versus being a female)
- Having graduated from a four-year college (versus having a high school degree or less)
- Identifying as a Democrat (versus identifying as a Republican)
- Having a family income of more than \$100,000 annually (versus having a family income of less than \$30,000 annually)
- Not having a religious affiliation (versus identifying as a Christian)
- Having CS or programming experience (versus not having such experience)


```{r demographicsupportregression, echo=FALSE, fig.height=10, fig.keep='all', cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Support for developing AI across demographic characteristics: average support across groups"}

# Clean the dataset
d_value_sum_support$demo_value <- factor(d_value_sum_support$demo_value, levels = rev(d_value_sum_support$demo_value))
shorten_sum <- function(x) {
  strsplit(as.character(x), split = ";")[[1]][1]  
}
                                             

# Generate graph
ggplot(data = d_value_sum_support, aes(x = demo_value, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_pointrange(position = position_dodge(width = 0.9), size = 0.25) + 
  geom_text(aes(y = num, label = roundfunc(num)), nudge_x = 0.3, nudge_y = 0.03,
            alpha = 0.6, size = 2.75) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
    name = "Demographic characteristics (grouped by demographic variable)") + 
  scale_y_continuous(
    name = "Support for developing AI (-2 = Strongly oppose; 2 = Strongly support)") + 
   facet_grid(demo_group~., scales = "free_y", space = "free_y") +
  labs(
       caption = "Source: Center for the Governance of AI") + 
  theme_bw() +
  theme(strip.background = element_blank(),
   strip.text.y = element_blank())

```

Some of the demographic differences we observe in this survey are in line with existing public opinion research. Below we highlight three salient predictors of support for AI based on the existing literature: gender, education, and income.  

Around the world, women have viewed AI more negatively than men. Fifty-four percent of women in EU countries viewed AI positively, compared with 67% of men [@eurobarometer460]. Likewise in the U.S., 44% of women perceived AI as unsafe -- compared with 30% of men [@morningconsult2017]. This gender difference could be explained by the fact that women have expressed higher distrust of technology than men do. In the U.S., women, compared with men, were more likely to view genetically modified foods or foods treated with pesticides as unsafe to eat, to oppose building more nuclear power plants, and to oppose fracking [@funk2015].

One's level of education also predicts one's enthusiasm toward AI, according to existing research. Reflecting upon their own jobs, 32% of Americans with no college education thought that technology had increased their opportunities to advance -- compared with 53% of Americans with a college degree [@smith2017]. Reflecting on the economy at large, 38% of those with post-graduate education felt that automation had helped American workers while only 19% of those with less than a college degree thought so [@graham2018]. A similar trend holds in the EU: those with more years of education, relative to those with fewer years, were more likely to value AI as good for society and less likely to think that AI steals people’s jobs [@eurobarometer460].

Another significant demographic divide in attitudes toward AI is income: low-income respondents, compared with high-income respondents, view AI more negatively. For instance, 40% of EU residents who had difficulty paying their bills "most of the time" hold negative views toward robots and AI, compared with 27% of those who "almost never" or "never" had difficulty paying their bills [@eurobarometer460]. In the U.S., 19% of those who made less than \$50,000 annually think that they are likely to lose their job to automation -- compared with only 8% of Americans who made more than \$100,000 annually [@graham2018]. Furthermore, Americans’ belief that AI will help the economy, as well as their support for AI research is positively correlated with their income [@morningconsult2017]. 

```{r demographicsupport2, echo=FALSE, fig.height=9, fig.keep='all', cache=TRUE, warning=FALSE, dpi = 300, fig.width=7, dev='pdf', fig.cap="Predicting support for developing AI using demographic characteristics: results from a multiple linear regression that includes all demographic variables"}

# Generate the data for the graph
dev_md_support_ai_o <- lm_robust(formula = as.formula("Q5_clean ~ demo_age + demo_gender + 
               demo_white + demo_educ + 
               demo_employ + 
    demo_pid3 + demo_income + demo_rel + demo_bornagain + demo_cs + 
    demo_prog"), data = d, weights = d$survey_weights)

dev_md_support_ai <- data.frame(variables = names(dev_md_support_ai_o$coefficients),
                     num = dev_md_support_ai_o$coefficients,
                     se = dev_md_support_ai_o$std.error,
                     p = dev_md_support_ai_o$p.value)
dev_md_support_ai$variables <- gsub(pattern = paste0(demo_var, collapse = "|"), replacement = "",
                         x = dev_md_support_ai$variables)
dev_md_support_ai$variables[dev_md_support_ai$variables == "Q5_clean"] <- 
  "Support for developing AI"

# # Make the stars
dev_md_support_ai$stars <- ""
dev_md_support_ai$stars[dev_md_support_ai$p < 0.05] <- "*"
dev_md_support_ai$stars[dev_md_support_ai$p < 0.01] <- "**"
dev_md_support_ai$stars[dev_md_support_ai$p < 0.001] <- "***"

# Make the text
dev_md_support_ai$new_text <- paste0(roundfunc(dev_md_support_ai$num), 
                      " (", roundfunc(dev_md_support_ai$se), 
                      ")", dev_md_support_ai$stars)


# add in the N and F-stat
dev_md_support_ai_k <- rbind(data.frame(variables = dev_md_support_ai$variables, coef = dev_md_support_ai$new_text),
                  data.frame(variables = "\\textit{N} = 2000",
                             coef = paste0("\\textit{F}(", round(dev_md_support_ai_o$fstatistic[2]), ",",
                           round(dev_md_support_ai_o$fstatistic[3]), ") = ",
                           roundfunc(dev_md_support_ai_o$fstatistic[1]), "; \\textit{p}-value: <0.001")))


# Fix the subgroup order 
dev_md_support_ai$variables <- factor(dev_md_support_ai$variables, 
                           levels = rev(dev_md_support_ai$variables[c(2:nrow(dev_md_support_ai), 1)]))

# Make a graph
ggplot(data = dev_md_support_ai[dev_md_support_ai$variables != "(Intercept)",], aes(x = variables, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_hline(yintercept = 0, linetype = 2, alpha = 0.5) +
  geom_pointrange(position = position_dodge(width = 0.9), size = 0.25) + 
  geom_text(aes(label = roundfunc(num)), nudge_x = 0.4, alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
                         name = "Demographic characteristics") + 
  scale_y_continuous( 
    name = "Coefficient estimates (outcome standardized)") + 
  expand_limits(x = c(1, nrow(dev_md_support_ai))) +
  labs(
       caption = "Source: Center for the Governance of AI") + 
  theme_bw()

```

## An overwhelming majority of Americans think that AI and robots should be carefully managed {#subsecsupportmanageai}

To compare Americans' attitudes with those of EU residents, we performed a survey experiment that replicated a question from the [2017 Special Eurobarometer #460](https://perma.cc/9FRT-ADST). (Details of the survey experiment are found in [Appendix B](#manageexp).) The original question asked respondents to what extent they agree or disagree with the following statement:

>Robots and artificial intelligence are technologies that require careful management.

We asked a similar question except respondents were randomly assigned to consider one of these three statements:

- AI and robots are technologies that require careful management.
- AI is a technology that requires careful management.
- Robots are technologies that require careful management. 

Our respondents were given the [same answer choices](#manageexp) presented to the Eurobarometer subjects.

The overwhelming majority of Americans -- more than eight in 10 -- agree that AI and/or robots should be carefully managed, while only 6% disagree, as seen in Figure \@ref(fig:aimanaged). [^airobotsfn] We find that variations in the statement wording produce [minor differences, statistically indistinguishable from zero,](#addcarefullym) in responses.

[^airobotsfn]: These percentages that we discuss here reflect the average response across the three statements. See [Appendix B](#manageexp) for the topline result for each statement. 

```{r aimanaged, echo=FALSE, fig.height=4.5, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, fig.width=7, dev='pdf', fig.cap="Agreement with statement that AI and/or robots should be carefully managed"}

# Overall mean for recoding missing values 
manage_ai_overall_mean <- wtd.mean(relabel_var(d$Q5b, c(1:5, 8, 9), 
                                              c(2, 1, -1, -2, NA, NA, NA)),
                            weights = d$survey_weights, na.rm = TRUE)
# Frequency table
manage_ai_value_table <- catvar_func(
  outcome = "Agreement that AI and robots should be carefully managed",
  outcome_var = d$Q5b,
  label_var = d0$Q5b,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 5,
  new_values <- c(2, 1, -1, -2, NA, NA, NA),
  missing_recode = manage_ai_overall_mean, 
  survey_weights = d$survey_weights
  )  
manage_ai_value_table$num <- c(2, 1, -1, -2, 8, 9)
# Numerical values
manage_ai_value_sum <- catvar_func(
  outcome = label(d0$Q5b),
  outcome_var = d$Q5b,
  label_var = d0$Q5b,
  output_type = "value_sum",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 5,
  new_values <- c(2, 1, -1, -2, NA, NA, NA),
  missing_recode = manage_ai_overall_mean,
  survey_weights = d$survey_weights
  )  

# Make the chart
manage_ai_value_table$labels <- factor(manage_ai_value_table$labels, 
                                       levels = manage_ai_value_table$labels[c(4:1, 5:6)])
ggplot() +
  geom_bar(data = manage_ai_value_table[manage_ai_value_table$Prop != 0,],
           aes(x = num, y = Prop, fill = labels),
           stat = "identity", alpha = 0.6, color = "black") +
  geom_errorbar(
    data =
      manage_ai_value_table[manage_ai_value_table$Prop != 0,],
    aes(
      x = num,
      ymin = Prop + qnorm(0.025) * se,
      ymax = Prop + qnorm(0.975) * se
    ),
    width = 0.1
  ) +
  geom_text(data = manage_ai_value_table,
            aes(x = num,
                label = roundfunc(Prop * 100, 0)),
            y = 0.03, nudge_x = 0.25) +
  scale_x_continuous(
    breaks = manage_ai_value_table$num[order(manage_ai_value_table$num)],
    labels = str_wrap(manage_ai_value_table$labels[order(manage_ai_value_table$num)],
                      width = 15)
  ) +
  facet_grid( ~ group, scales = "free_x", space = "free_x") + theme_bw() +
  geom_text(data = manage_ai_value_sum, aes(
    x = 0,
    label = sum_stat,
    y = max(manage_ai_value_table$Prop) +
      0.05
  )) +
  scale_y_continuous(labels = scales::percent,
                     limits = c(0, max(manage_ai_value_table$Prop) + 0.05)) +
      scale_fill_manual(values = c("#ca0020", "#f4a582", "#92c5de", "#0571b0",
                               "grey65")) + 
  xlab("Responses") + ylab("Percentage of respondents") +
  labs(caption = "Source: Center for the Governance of AI") + theme(legend.position = "none")

```

```{r aimanagedexp, echo=FALSE, fig.height=5, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.width=7}
# Clean the data
d$Q5b_clean <- relabel_var(d$Q5b, c(1:5, 8, 9), c(2, 1, -1, -2, NA, NA, NA))
d$Q5b_missing <- is.na(d$Q5b_clean)
d$Q5b_clean[is.na(d$Q5b_clean)] <- wtd.mean(d$Q5b_clean[!is.na(d$Q5b_clean)], 
                                            weights = d$survey_weights[!is.na(d$Q5b_clean)])
d$q5b_treat_clean <- relabel_var(d$q5b_treat, c(1:3, 8, 9), 
                                 c("AI and robots", "AI", "Robots", NA, NA))

manage_func <- function(exp_group) {
  data.frame(catvar_func(
  outcome = label(d0$Q5b),
  outcome_var = d$Q5b,
  label_var = d0$Q5b,
  output_type = "value_sum",
  shown = d$q5b_treat_clean == exp_group,
  num_missing = 8,
  num_DK = 5,
  new_values <- c(2, 1, -1, -2, NA, NA, NA),
  missing_recode = manage_ai_overall_mean,
  survey_weights = d$survey_weights
  ), q5b_treat_clean = exp_group)
}
tech_manage <- lapply(c("AI and robots", "AI", "Robots"), manage_func) %>% 
  do.call(what = rbind)
tech_manage$q5b_treat_clean <- factor(tech_manage$q5b_treat_clean, 
                                      levels = c("AI", "Robots", "AI and robots"))
```

```{r aimanagedexp2, echo=FALSE, fig.height=3.5, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.width=7, fig.cap="Agreement with statement that AI and/or robots should be carefully managed by experimental condition"}
# Make the plot
ggplot(data = tech_manage, aes(x = q5b_treat_clean, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_pointrange(position = position_dodge(width = 0.9)) + 
  geom_text(aes(label = sum_stat), nudge_x = 0.3, alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 20),
                         name = "Experimental groups", expand = c(0.1, 0)) + 
  scale_y_continuous(expand = c(0.05, 0.05),
    name = "Agreement/disagreement with statement\n(-2 = Totally disagree; 2 = Totally agree)") + 
  expand_limits(x = c(1, 4)) +
  labs(caption = "Source: Center for the Governance of AI") + theme_bw()

```

Next, we compared our survey results with the responses from the 2017 Special Eurobarometer #460 by country [@eurobarometer460]. For the U.S., we used all the responses to our survey question, unconditional on the experimental condition, because the variations in question-wording do not affect responses.

The percentage of those in the U.S. who agree with the statement (82%) is not far off from the EU average (88% agreed with the statement). Likewise, the percentage of Americans who disagree with the statement (6% disagree) is comparable with the EU average (7% disagreed). The U.S. ranks among the lowest regarding the agreement with the statement in part due to the relatively high percentage of respondents who selected the "don't know" option. 

```{r eu, echo=FALSE, fig.height=8, fig.keep='all', fig.width=7, dev='pdf', dpi=300, warning=FALSE, cache=TRUE, fig.cap="Agreement with statement that robots and AI require careful management (EU data from 2017 Special Eurobarometer #460)"}
# Get US data
us_ai_manage <- catvar_func(
    outcome = label(x = d0$Q5b),
    outcome_var = d$Q5b,
    label_var = d0$Q5b,
    output_type = "value_table",
    shown = rep(TRUE, nrow(d)),
    num_missing = 8,
    num_DK = 5, missing_recode = manage_ai_overall_mean,
    new_values <- c(2, 1, -1, -2, NA, NA, NA))

# Load Eurobarometer data
euro <- read.csv(paste0(wd, "eb_2017_qd12_3.csv"), 
                 stringsAsFactors=FALSE)
euro$group <- "No highlight"
euro$group[euro$Country == "EU"] <- "Highlight"
euro_com <- rbind(euro, data.frame(Country = "US", 
           Totally_agree = us_ai_manage$Prop[us_ai_manage$num == 1]*100,
           Tend_to_agree = us_ai_manage$Prop[us_ai_manage$num == 2]*100,
           Tend_to_disagree = us_ai_manage$Prop[us_ai_manage$num == 3]*100,
           Totally_disagree = us_ai_manage$Prop[us_ai_manage$num == 4]*100,
           Dont_know = us_ai_manage$Prop[us_ai_manage$num == 5]*100,
           Total_agree = sum(us_ai_manage$Prop[us_ai_manage$num %in% c(1, 2)])*100,
           Total_disagree = sum(us_ai_manage$Prop[us_ai_manage$num %in% c(3, 4)])*100,
           group = "Highlight")) %>% reshape2::melt(id = c("Country", "group"))
euro_responses <- c("Totally agree", "Tend to agree", "Tend to disagree",
                                   "Totally disagree", "Don't know", "Total agree",
                                   "Total disagree")
euro_com$variable <- relabel_var(euro_com$variable, old_labels = unique(euro_com$variable),
                                 new_labels = euro_responses)
euro_com$variable <- factor(euro_com$variable, euro_responses[c(5, 4, 3, 2, 1, 6, 7)])
euro_com$stacked <- !euro_com$variable %in% c("Total agree", "Total disagree")


# Make the graph
euro_com$iso2c <- euro_com$Country
euro_com <- merge(x = euro_com, y = codelist[,c("cow.name", "iso2c")], all.x = TRUE)
euro_com$cow.name[euro_com$Country == "EU"] <- "European Union"
euro_com$cow.name[euro_com$Country == "UK"] <- "United Kingdom"
euro_com$cow.name[euro_com$Country == "EL"] <- "Greece"
euro_com$cow.name[euro_com$Country == "US"] <- "United States"
euro_com$cow.name <- factor(euro_com$cow.name,
                           levels = euro_com$cow.name[euro_com$variable == "Total agree"][order(euro_com$value[euro_com$variable == "Total agree"])])

ggplot(data = euro_com[euro_com$stacked, ], aes(x = cow.name, y = value /
                                                  100, fill = variable)) +
  geom_bar(stat = "identity", alpha = 0.8) +
  geom_vline(xintercept = c(2.5, 3.5, 12.5, 13.5), 
             linetype = 2, size = 0.75) + 
  scale_fill_manual(
    values = c("grey65", "#ca0020", "#f4a582", "#92c5de", "#0571b0"),
    labels = rev(
      c(
        "2. Totally agree",
        "1. Tend to agree",
        "-1. Tend to disagree",
        "-2. Totally disagree",
        "Don't know"
      )
    ),
    name = "Responses"
  ) + 
  xlab("Countries") +
  scale_y_continuous(
    name = "Percentage of respondents",
    labels = scales::percent,
    limits = c(0, 1),
    expand = c(0, 0)
  ) + coord_flip() + 
  theme_bw() + theme(legend.position = "bottom",
                     axis.text.x = 
                       element_text(hjust = 1)) +
  guides(fill = guide_legend(reverse = TRUE, nrow = 2)) +
  labs(caption = "Source: Center for the Governance of AI; Eurobarometer")

# Make the table for the Appendix
euro_com_w <- spread(euro_com[euro_com$stacked,-which(names(euro_com) %in% 
                                                        c("group", "stacked", "iso2c",
                                                          "Country"))], 
                     key = variable, value = value)[,c(1, 3:6, 2)]
euro_com_w$cow.name <- as.character(euro_com_w$cow.name)
euro_com_w[,-1] <- round(euro_com_w[,-1])
euro_com_w <- euro_com_w[order(euro_com_w$cow.name),]
```

## Harmful consequences of AI in the context of other global risks 

```{r globalrisksfig, echo=FALSE, fig.height=7.5, fig.keep='all', fig.cap="The American public's perceptions of 15 potential global risks", warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}

global_risks <- c("Failure to address climate change", 
                  "Failure of regional/global governance",
                  "Conflict between major countries", 
                  "Weapons of mass destruction",
                  "Large-scale involuntary migration", 
                  "Spread of infectious diseases", 
                  "Water crises",
                  "Food crises",
                  "Harmful consequences of AI", 
                  "Harmful consequences of synthetic biology",
                  "Cyber attacks",
                  "Terrorist attacks",
                  "Global recession", 
                  "Extreme weather events", 
                  "Natural disasters")


global_risks_clean <- function(risk_num) {
  # Convert multiple-choice outcomes to slider outcomes 
  mc_outcome <- relabel_var(d[,paste0("Q1_", risk_num)], 
                            old_labels = c(1:8, 98, 99), 
                            new_labels = c(mc_p_med, NA, NA, NA))
  # Clean the impact outcomes
  impact <- d[,paste0("Q2_", risk_num)] 
  impact <- relabel_var(impact, old_labels = c(1:6, 8, 9), 
                        new_labels = c(0, 1, 2, 3, 4, NA, NA, NA))
  # Make into a dataframe
  temp <- data.frame(respondent_id = d$r_id,
               risk = global_risks[risk_num],
               survey_weights = d$survey_weights,
                    prob = mc_outcome,
                    impact = impact)
  # Remove the respondents who aren't show the risk
  return(temp[!is.na(d[,paste0("Q1_risks_", risk_num)]) &
                       d[,paste0("Q1_risks_", risk_num)] == 1,])
}

# Clean up the data
gr_clean <- lapply(1:15, global_risks_clean) %>% do.call(what = rbind)
# Recode the missing values
gr_clean$prob_missing <- is.na(gr_clean$prob)
gr_clean$impact_missing <- is.na(gr_clean$impact)
gr_clean$prob[is.na(gr_clean$prob)] <- 
  wtd.mean(gr_clean$prob, weights = gr_clean$survey_weights, na.rm = TRUE)
gr_clean$impact[is.na(gr_clean$impact)] <- 
  wtd.mean(gr_clean$impact, weights = gr_clean$survey_weights, na.rm = TRUE)
# Check the percent missing across risk
gr_sum_missing <- gr_clean %>% group_by(risk) %>% dplyr::summarise(
  prob_missing = mean(prob_missing),
  impact_missing = mean(impact_missing)
)

# Summarize the data
# Helper function
gr_sum_func <- function(risk) {
  md_prob <- if (gr_sum_missing$prob_missing[gr_sum_missing$risk == risk] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
    md <- lm(prob ~ scale(prob_missing), data = gr_clean[gr_clean$risk == risk,],
               weights = gr_clean$survey_weights[gr_clean$risk == risk])
    coeftest(md, vcov = vcovHC(md, type="HC2"))
    } else {
      md <- lm(prob ~ 1, 
                 weights = gr_clean$survey_weights[gr_clean$risk == risk], 
                 data = gr_clean[gr_clean$risk == risk,])
      coeftest(md, vcov = vcovHC(md, type="HC2"))
    }
  md_impact <- if (gr_sum_missing$impact_missing[gr_sum_missing$risk == risk] > 0.1) {
    # If more than 10 percent is missing, then we condition on the normalized dummy variable for missingness
      md <- lm(impact ~ scale(impact_missing), 
                 data = gr_clean[gr_clean$risk == risk,],
               weights = gr_clean$survey_weights[gr_clean$risk == risk])
      coeftest(md, vcov = vcovHC(md, type="HC2"))
    } else {
      md <- lm(impact ~ 1, weights = gr_clean$survey_weights[gr_clean$risk == risk], 
                 data = gr_clean[gr_clean$risk == risk,])
      coeftest(md, vcov = vcovHC(md, type="HC2"))
    }
  data.frame(risk = risk, prob = md_prob[1,1], impact = md_impact[1,1], 
             N = length(gr_clean$respondent_id[gr_clean$risk == risk]))
}

# Run the analysis
gr_sum <- lapply(global_risks, gr_sum_func) %>% do.call(what = rbind)
# Make the confidence ellpises
gr_cr_func <- function(risk, alpha = 0.05, m = 2000) {
  survey_weights <- gr_clean$survey_weights[gr_clean$risk == risk]
  md_prob <- if (gr_sum_missing$prob_missing[gr_sum_missing$risk == risk] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      lm(prob ~ scale(prob_missing), data = gr_clean[gr_clean$risk == risk,],
               weights = survey_weights)
    } else {
      lm(prob ~ 1, 
                 weights = survey_weights, 
                 data = gr_clean[gr_clean$risk == risk,])
    }
  md_impact <- if (gr_sum_missing$impact_missing[gr_sum_missing$risk == risk] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      lm(impact ~ scale(impact_missing), 
                 data = gr_clean[gr_clean$risk == risk,],
               weights = survey_weights)
    } else {
      lm(impact ~ 1, weights = survey_weights, 
                 data = gr_clean[gr_clean$risk == risk,])
    }
  # Generate the variance-covariance matrix using the residuals from our models
  cov_res <- cov(cbind(md_prob$residuals, md_impact$residuals))
  # Generate data for the confidence ellipse
  bivCI_res <- bivCI(s = cov_res, xbar = c(md_prob$coefficients[1],
                                               md_impact$coefficients[1]),
      n = length(survey_weights), alpha = alpha, m = m)
  return(data.frame(risk = risk, prob = bivCI_res$x, impact = bivCI_res$y))
}
gr_cr <- do.call(rbind, lapply(global_risks, gr_cr_func))
# Plot the data
ggplot() + 
  geom_path(data = gr_cr, aes(x = prob/100, y = impact, color = risk), alpha = 0.3) + 
  geom_point(data = gr_sum, aes(x = prob/100, y = impact, color = risk), size = 3) +
  geom_text_repel(data = gr_sum, aes(x = prob/100, y = impact,
                                     label = str_wrap(risk, width = 20)), 
                  size = 3) +
  scale_x_continuous(name = "Likelihood of happening within 10 years (percentage points)",
                     labels = scales::percent) + 
  ylab("Impact (0 = minimal; 4= catastrophic)") +
  theme_bw() + theme(legend.position="none") +
  labs(
       caption = "Source: Center for the Governance of AI")
```

At the beginning of the survey, respondents were asked to consider five out of 15 potential global risks (the descriptions are found in [Appendix B](#global_risks)). The purpose of this task was to compare respondents' perception of AI as a global risk with their notions of other potential global risks. The global risks were selected from the [Global Risks Report 2018](https://perma.cc/8XM8-LKEN), published by the World Economic Forum. We edited the description of each risk to be more comprehensible to non-expert respondents while preserving the substantive content. We gave the following definition for a global risk:

>A "global risk" is an uncertain event or condition that, if it happens, could cause significant negative impact for at least 10 percent of the world's population. That is, at least 1 in 10 people around the world could experience a significant negative impact. [^globalriskfn]

[^globalriskfn]: Our definition of global risk borrowed from the Global Challenges Foundation's definition: "an uncertain event or condition that, if it happens, can cause a significant negative impact on at least 10% of the world's population within the next 10 years" [@cotton2016].

After considering each potential global risk, respondents were asked to evaluate the likelihood of it happening globally within 10 years, as well as its impact on several countries or industries.

We use a scatterplot (Figure \@ref(fig:globalrisksfig) to visualize results from respondents' evaluations of global risks. The _x_-axis is the perceived likelihood of the risk happening globally within 10 years. The _y_-axis is the perceived impact of the risk. The mean perceived likelihood and impact is represented by a dot. The corresponding ellipse contains the 95% confidence region.

In general, Americans perceive all these risks to be impactful: on average they rate each as having between a moderate (2) and severe (3) negative impact if they were to occur. Americans perceive the use of weapons of mass destruction to be the most impactful -- at the "severe" level (mean score 3.0 out of 4). Although they do not think this risk as likely as other risks, they still assign it an average of 49% probability of occurring within 10 years. Risks in the upper-right quadrant are perceived to be the most likely as well as the most impactful. These include natural disasters, cyber attacks, and extreme weather events.   

The American public and the nearly 1,000 experts surveyed by the World Economic Forum share similar views regarding most of the potential global risks we asked about [@wef2018]. Both the public and the experts rank extreme weather events, natural disasters, and cyber attacks as the top three most likely global risks; likewise, both groups consider weapons of mass destruction to be the most impactful. Nevertheless, compared with experts, Americans offer a lower estimate of the likelihood and impact of the failure to address climate change. 

The American public appears to over-estimate the likelihoods of these risks materializing within 10 years. The mean responses suggest (assuming independence) that about eight (out of 15) of these global risks, which would have a significant negative impact on at least 10% of the world's population, will take place in the next 10 years. One explanation for this is that it arises from the broad misconception that the world is in a much worse state than it is in reality [@pinker2018enlightenment; @rosling2018factfulness]. Another explanation is that it arises as a byproduct of respondents interpreting "significant negative impact" in a relatively minimal way, though this interpretation is hard to sustain given the mean severity being between "moderate" and "severe." Finally, this result may be because subjects centered their responses within the distribution of our response options, the middle value of which was the 40-60% option; thus, the likelihoods should not be interpreted literally in the absolute sense. 

The adverse consequences of AI within the next 10 years appear to be a relatively low priority in respondents' assessment of global risks. It -- along with adverse consequences of synthetic biology -- occupy the lower left quadrant, which contains what are perceived to be lower-probability, lower-impact risks. [^weffn] These risks are perceived to be as impactful (within the next 10 years) as the failure to address climate change, though less probable. One interpretation of this is that the average American simply does not regard AI as posing a substantial global risk. This interpretation, however, would be in tension with some expert assessment of catastrophic risks that suggests unsafe AI could pose significant danger [@wef2018; @sandberg2008]. The gap between experts and the public's assessment suggests that this is a fruitful area for efforts to educate the public.

Another interpretation of our results is that Americans do have substantial concerns about the long-run impacts of advanced AI, but they do not see these risks as likely in the coming 10 years. As support for this interpretation, we later find that 12% of American's believe the impact of high-level machine intelligence will be "extremely bad, possibly human extinction," and 21% that it will be "on balance bad." Still, even though the median respondent expects around a 54% chance of high level machine intelligence within 10 years, respondents may believe that the risks from high level machine intelligence will manifest years later. If we assume respondents believe global catastrophic risks from AI only emerge from high-level AI, we can infer an implied global risk, conditional on high-level AI (within 10 years), of 80%. Future work should try to unpack and understand these beliefs.

We used a survey experiment to understand how the public understands the terms _AI_, _automation_, _machine learning_, and _robotics_. (Details of the survey experiment are found in [Appendix B](#considersai).) We randomly assigned each respondent one of these terms and asked them:

>In your opinion, which of the following technologies, if any, uses [artificial intelligence (AI)/automation/machine learning/robotics]? Select all that apply.

Because we wanted to understand respondents' perceptions of these terms, we did not define any of the terms. Respondents were asked to consider [10 technological applications](#considersai), each of which uses AI or machine learning. 

Though the respondents show at least a partial understanding of the terms and can identify their use within the considered technological applications correctly, the respondents underestimate the prevalence of AI, machine learning, and robotics in everyday technological applications, as reported in Figure \@ref(fig:whatai). (See [Appendix C](#aawhatsai) for details of our statistical analysis.)

Among those assigned the term _AI_, a majority think that virtual assistants (63%), smart speakers (55%), driverless cars (56%), social robots (64%), and autonomous drones use AI (54%). Nevertheless, a majority of respondents assume that Facebook photo tagging, Google Search, Netflix or Amazon recommendations, or Google Translate do not use AI.

Why did so few respondents consider the products and services we listed to be applications of AI, automation, machine learning, or robotics? 

```{r whatai, echo=FALSE, fig.height=9.5, fig.keep='all', fig.cap="What applications or products that the public thinks use AI, automation, machine learning, or robotics", fig.width=7, cache=TRUE, warning=FALSE, dpi = 300, dev='pdf'}

# Helper function 
whatsai <- function(technum, output_type = "num_outcome") {
  catvar_func(
  outcome = label(d0[,paste0("Q3new_", technum)]),
  outcome_var = d[,paste0("Q3new_", technum)],
  label_var = d0[,paste0("Q3new_", technum)],
  output_type = output_type,
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = -99, 
  new_values <- c(1, 0, NA, NA), 
  survey_weights = d$survey_weights, 
  missing_recode = wtd.mean(relabel_var(d[,paste0("Q3new_", technum)],
                                        c(1, 2, 8, 9), c(1, 2, NA, NA)), 
                            weights = d[,"survey_weights"], na.rm = TRUE)
  )  
}
# Run the analysis 
whatsai_d <- data.frame(techtreat = d$q3new_treat,
                        survey_weights = d$survey_weights,
                        do.call(cbind, lapply(1:10, whatsai)))
# Generate summary statistics by treatment and application
whatsai_d <- reshape2::melt(whatsai_d, id = c("techtreat", "survey_weights")) %>%
  group_by(techtreat, variable) %>% dplyr::summarise(
    prop = md_weight(value, weights = survey_weights, 
                     which_stat = "mean"),
    se = md_weight(value, weights = survey_weights, which_stat = "se"),
    N = n())
# Label the treatments
whatsai_d$techtreat <- relabel_var(old_var = whatsai_d$techtreat, old_labels = c(1:4),
            new_labels = c("Artificial intelligence (AI)", 
                           "Automation", "Machine learning", "Robotics"))
whatsai_d$variable <- relabel_var(old_var = whatsai_d$variable, 
                                  old_labels = levels(whatsai_d$variable),
            new_labels = label(d0)[paste0("Q3new_", 1:10)]) %>% 
  factor(levels = rev(label(d0)[paste0("Q3new_", 1:10)]))

# Make the graph
whatsai_d$techtreat <- factor(whatsai_d$techtreat, levels = rev(unique(whatsai_d$techtreat)))
whatsai_d$variable <- factor(whatsai_d$variable, levels = unique(whatsai_d$variable))

# Make the graph
whatsai_d_graph <- whatsai_d
ggplot(whatsai_d) +
  geom_bar(aes(x = techtreat, y = prop, fill = techtreat), 
           stat = "identity") + 
  geom_errorbar(data = 
                  whatsai_d[whatsai_d$prop !=0,], 
                aes(x = techtreat, ymin = prop + qnorm(0.025)*se,
                    ymax = prop + qnorm(0.975)*se), width = 0.1) +
    geom_text(aes(x = techtreat, y = 0.05, label = round(prop*100))) + 
  coord_flip() +
  scale_y_continuous(labels = scales::percent, 
  name = "Percentage of respondents who selected the term", 
                     limits = c(0, 1), expand = c(0,0)) +
  scale_x_discrete(labels = function(x) str_wrap(x, width = 40), 
                   name = "Technology terms") +
  scale_fill_discrete(name = "Technology terms") +
  facet_wrap(~variable, ncol = 2,
             labeller = label_wrap_gen(width = 45)) +
  theme_bw() +
  theme(legend.position = "bottom", legend.direction = "vertical",
        axis.text.y=element_blank(),
        axis.ticks.y=element_blank(),
        axis.text.x = element_text(hjust=1),
        panel.spacing.x = unit(1.5, "lines")) + 
  guides(fill = guide_legend(reverse = TRUE, nrow = 2)) +
  labs(caption = "Source: Center for the Governance of AI")

```

A straightforward explanation is that inattentive respondents neglect to carefully consider or select the items presented to them (i.e., non-response bias). Even among those assigned the term _robotics_, only 62% selected social robots and 68% selected industrial robots. Our analysis (found in [Appendix C](#aawhatsai)) confirms that respondent inattention, defined as spending too little or too much time on the survey, predicts non-response to this question.

Another potential explanation for the results is that the American public -- like the public elsewhere -- lack awareness of AI or machine learning. As a result, the public does not know that many tech products and services use AI or machine learning. According to a 2017 survey, nearly half of Americans reported that they were unfamiliar with AI [@morningconsult2017]. In the same year, only 9% of the British public said they had heard of the term "machine learning" [@rs2018]. Similarly, less than half of EU residents reported hearing, reading, or seeing something about AI in the previous year [@eurobarometer460].

Finally, the so-called "AI effect" could also explain the survey result. The AI effect describes the phenomenon that the public does not consider an application that uses AI to utilize AI once that application becomes commonplace [@McCorduck2004]. Because 85% of Americans report using digital products that deploy AI (e.g., navigation apps, video or music streaming apps, digital personal assistants on smartphones, etc.)  [@reinhart2018], they may not think that these everyday applications deploy AI. 

\newpage

# Public opinion on AI governance

## Americans consider many AI governance challenges to be important; prioritize data privacy and preventing AI-enhanced cyber attacks, surveillance, and digital manipulation {#subsecgovchallenges13}

We sought to understand how Americans prioritize policy issues associated with AI. Respondents were asked to consider five AI governance challenges, randomly selected from a test of 13 ([see Appendix B for the text](#govchallenges)); the order these five were to each respondent was also randomized. 

After considering each governance challenge, respondents were asked how likely they think the challenge will affect large numbers of people 1) in the U.S. and 2) around the world within 10 years. 

We use scatterplots to visualize our survey results. In Figure \@ref(fig:airisksus), the _x_-axis is the perceived likelihood of the problem happening to large numbers of people in the U.S. In Figure \@ref(fig:airisksworld), the _x_-axis is the perceived likelihood of the problem happening to large numbers of people around the world. The _y_-axes on both Figure \@ref(fig:airisksus) and \@ref(fig:airisksworld) represent respondents' perceived issue importance, from 0 (not at all important) to 3 (very important). Each dot represents the mean perceived likelihood and issue importance, and the correspondent ellipse represents the 95% bivariate confidence region. 

Americans consider all the AI governance challenges we present to be important: the mean perceived issues importance of each governance challenge is between "somewhat important" (2) and "very important" (3), though there is meaningful and discernible variation across items. 

The AI governance challenges Americans think are most likely to impact large numbers of people, and are important for tech companies and governments to tackle, are found in the upper-right quadrant of the two plots. These issues include data privacy as well as AI-enhanced cyber attacks, surveillance, and digital manipulation. We note that the media have widely covered these issues during the time of the survey.

There are a second set of governance challenges that are perceived on average, as about 7% less likely, and marginally less important. These include autonomous vehicles, value alignment, bias in using AI for hiring, the U.S.-China arms race, disease diagnosis, and technological unemployment. Finally, the third set of challenges are perceived on average another 5% less likely, and about equally important, including criminal justice bias and critical AI systems failures. 

We also note that Americans predict that all of the governance challenges mentioned in the survey, besides protecting data privacy and ensuring the safety of autonomous vehicles, are more likely to impact people around the world than to affect people in the U.S. While most of the statistically significant differences are substantively small, one difference stands out: Americans think that autonomous weapons are 7.6 percentage points more likely to impact people around the world than Americans. (See [Appendix C](#appgovchallenges) for details of these additional analyses.) 

We want to reflect on one result. "Value alignment" consists of an abstract description of alignment problem and a reference to what sounds like individual level harms: "while performing jobs [they could] unintentionally make decisions that go against the values of its human users, such as physically harming people." "Critical AI systems failures," by contrast, references military or critical infrastructure uses, and unintentional accidents leading to "10 percent or more of all humans to die." The latter was weighted as less important than the former: we interpret this as a probability weighted assessment of importance, since presumably the latter, were it to happen, is much more important. We thus think the issue importance question should be interpreted in a way that down-weights low probability risks. This perspective also plausibly applies to the "impact" measure for our global risks analysis, which placed "harmful consequences of synthetic biology" and "failure to address climate change" as less impactful than most other risks. 

```{r airisks, echo=FALSE, fig.height=5, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.width=7}

# Clean up data

# Labels for the government challenges
ai_gov <-
  c(
  "Hiring bias",
  "Criminal justice bias",
  "Disease diagnosis",
  "Data privacy",
  "Autonomous vehicles",
  "Digital manipulation",
  "Cyber attacks",
  "Surveillance",
  "U.S.-China arms race",
  "Value alignment",
  "Autonomous weapons",
  "Technological unemployment",
  "Critical AI systems failure"
  )


var_name = "Q8_"

# Helper function 
ai_gov_clean <- function(risk_num, var_name) {
  # Convert multiple-choice outcomes to slider outcomes 
  mc_outcome <- relabel_var(d[,paste0(var_name, risk_num)], 
                            old_labels = c(1:8, 98, 99), 
                            new_labels = c(mc_p_med, NA, NA, NA))
  # Clean the importance outcomes
  importance <- d[,paste0("Q10_", risk_num)] 
  importance <- relabel_var(importance, old_labels = c(1:5, 8, 9), 
                        new_labels = c(3, 2, 1, 0, NA, NA, NA))
  # Make into a dataframe
  temp <- data.frame(respondent_id = d$r_id,
               gov_challenge = rep(ai_gov[risk_num], nrow(d)),
                    prob = mc_outcome,
                    importance = importance,
               survey_weights = d$survey_weights,
               demo_age = d$demo_age,
               demo_gender = d$demo_gender,
               demo_white = d$demo_white,
               demo_educ = d$demo_educ,
               demo_employ = d$demo_employ,
               demo_income = d$demo_income,
               demo_pid3 = d$demo_pid3,
               demo_rel = d$demo_rel,
               demo_bornagain = d$demo_bornagain,
               demo_cs = d$demo_cs,
               demo_prog = d$demo_prog,
               num_year = d$birthyr
               )
  # Remove the respondents who aren't show the risk
  return(temp[!is.na(d[,paste0("Q8_challenge_", risk_num)]) &
                       d[,paste0("Q8_challenge_", risk_num)] == 1,])
}

# Clean up the data
# US

ag_clean_US <- do.call(rbind, 
                       lapply(1:13, ai_gov_clean, var_name = "Q8_"))

# Recode the missing values
ag_clean_US$prob_missing <- is.na(ag_clean_US$prob)
ag_clean_US$importance_missing <- is.na(ag_clean_US$importance)
ag_clean_US$prob[is.na(ag_clean_US$prob)] <- wtd.mean(ag_clean_US$prob, 
                                                      weights = ag_clean_US$survey_weights,
                                                      na.rm =  TRUE)
ag_clean_US$importance[is.na(ag_clean_US$importance)] <- wtd.mean(ag_clean_US$importance, 
                                                      weights = ag_clean_US$survey_weights,
                                                      na.rm =  TRUE)
ag_clean_US$geo <- "U.S."

# World 
ag_clean_world <- do.call(rbind, lapply(1:13, ai_gov_clean, var_name = "Q9_"))
# Recode the missing values
ag_clean_world$prob_missing <- is.na(ag_clean_world$prob)
ag_clean_world$importance_missing <- is.na(ag_clean_world$importance)
ag_clean_world$prob[is.na(ag_clean_world$prob)] <- 
  wtd.mean(ag_clean_world$prob, weights = ag_clean_world$survey_weights,
                                                      na.rm =  TRUE)
ag_clean_world$importance[is.na(ag_clean_world$importance)] <- 
  wtd.mean(ag_clean_world$importance, weights = ag_clean_world$survey_weights,
                                                      na.rm =  TRUE)
ag_clean_world$geo <- "World"
# Check the percent of missing data
ag_clean_all <- rbind(ag_clean_US, ag_clean_world)
ag_missing <- ag_clean_all %>% group_by(geo, gov_challenge) %>% dplyr::summarise(
  prob_missing = mean(prob_missing),
  importance_missing = mean(importance_missing)
) %>% as.data.frame()

# Summarize the data
# Helper function
ag_sum_func <- function(ag, geo) {
  survey_weights <- ag_clean_all$survey_weights[ag_clean_all$gov_challenge == ag &
                                                   ag_clean_all$geo == geo]
  # Likelihood 
  md_prob <- if (ag_missing$prob_missing[ag_missing$gov_challenge == ag &
                                         ag_missing$geo == geo] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      md <- lm(prob ~ scale(prob_missing), 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
               weights = survey_weights)
      coeftest(md, vcov = vcovHC(md, type="HC2"))
    } else {
      md <- lm(prob ~ 1, 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
               weights = survey_weights)
      coeftest(md, vcov = vcovHC(md, type="HC2"))
    }
  # Issue importance
  md_importance <- if (ag_missing$importance_missing[ag_missing$gov_challenge == ag &
                                         ag_missing$geo == geo] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      md <- lm(importance ~ scale(importance_missing), 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
               weights = survey_weights)
    coeftest(md, vcov = vcovHC(md, type="HC2"))
    } else {
      md <- lm(importance ~ 1, 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
               weights = survey_weights)
      coeftest(md, vcov = vcovHC(md, type="HC2"))
    }
  data.frame(gov_challenge = ag, prob = md_prob[1,1], importance = md_importance[1,1],
             prob_se = md_prob[1,2], importance_se = md_importance[1,2],
             N = length(survey_weights),
             region = geo)
}
# Run the analysis
ag_sum_US <- lapply(ai_gov, ag_sum_func, geo = "U.S.") %>% do.call(what = rbind)
ag_sum_world <- lapply(ai_gov, ag_sum_func, geo = "World") %>% do.call(what = rbind)
# Combine the U.S. and world results
ag_sum_all <- rbind(ag_sum_US, ag_sum_world)
# Function to get the confidence ellipses
ag_cr_func <- function(ag, geo, alpha = 0.5, m = 2000) {
   # Likelihood 
  survey_weights <- ag_clean_all$survey_weights[ag_clean_all$gov_challenge == ag &
                                                   ag_clean_all$geo == geo]
  md_prob <- if (ag_missing$prob_missing[ag_missing$gov_challenge == ag &
                                         ag_missing$geo == geo] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      lm(prob ~ scale(prob_missing), 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
         weights = survey_weights)
    } else {
      lm(prob ~ 1, 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
         weights = survey_weights)
    }
  # Issue importance
  md_importance <- if (ag_missing$importance_missing[ag_missing$gov_challenge == ag &
                                         ag_missing$geo == geo] > 0.1) {
    # If more than 10 percent is missing, then we condition on normalized dummy variable for missingness
      lm(importance ~ scale(importance_missing), 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
               weights = survey_weights)
    } else {
      lm(importance ~ 1, 
                 data = ag_clean_all[ag_clean_all$gov_challenge == ag &
                                   ag_clean_all$geo == geo,],
               weights = survey_weights)
    }
  # Generate the variance-covariance matrix using the residuals from our models
  cov_res <- cov(cbind(md_prob$residuals, md_importance$residuals))
  # Generate data for the confidence ellipse
  bivCI_res <- bivCI(s = cov_res, xbar = c(md_prob$coefficients[1],
                                               md_importance$coefficients[1]),
      n = length(survey_weights), alpha = alpha, m = m)
  return(data.frame(gov_challenge = ag, prob = bivCI_res$x, importance = bivCI_res$y))
}
ag_cr_US <- do.call(rbind, lapply(ai_gov, ag_cr_func, geo = "U.S."))
ag_cr_US$region <- "U.S."
ag_cr_world <- do.call(rbind, lapply(ai_gov, ag_cr_func, geo = "World"))
ag_cr_world$region <- "World"
ag_cr_all <- rbind(ag_cr_US, ag_cr_world)
```

```{r airisksus, echo=FALSE, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.cap="Perceptions of AI governance challenges in the U.S.", out.extra='angle=270', fig.height=6, fig.width=9, out.width='8in', fig.align='center'}

# Function to test the difference
diff_us_world <- function(gov_challenge) {
  ag_data <- ag_clean_all[ag_clean_all$gov_challenge == 
                         gov_challenge,]
ag_md <- lm(prob ~ geo, 
   data = ag_data,
   weights = ag_data$survey_weights)
coef_res <- coef_test(ag_md, vcov = "CR2", 
          cluster = ag_data$respondent_id, test = "naive-t")
return(data.frame(gov_challenge = gov_challenge,
                  mean_us = coef_res[1,1],
                     num = coef_res[2,1],
                     se = coef_res[2,2],
                     pvalue = coef_res[2,3]))
}

# Difference
ai_gov_uw_diff <- do.call(rbind, lapply(ai_gov, diff_us_world))

# Plot the US data
ggplot() + 
  geom_path(data = ag_cr_US, aes(x = prob/100, y = importance, color = gov_challenge),
            alpha = 0.3) + 
  geom_point(data = ag_sum_US, aes(x = prob/100, y = importance, 
                                color = gov_challenge), size = 3) +
  geom_text_repel(data = ag_sum_US, aes(x = prob/100, y = importance, 
                                     label = str_wrap(gov_challenge, width = 20)), 
                  point.padding = 0.75, segment.alpha = 0.6) +
  scale_x_continuous(name = "Likelihood of impacting large numbers of people in the U.S. within 10 years", labels = scales::percent, limits = c(0.50, 0.725)) + 
  ylab("Issue importance\n(0 = Not at all important; 3 = Very important)") +
  theme_bw() + theme(legend.position="none") + 
  labs(
       caption = "Source: Center for the Governance of AI")
```

```{r airisksworld, echo=FALSE, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.cap="Perceptions of AI governance challenges around the world", out.extra='angle=270', fig.height=6, fig.width=9, out.width='8in', fig.align='center'}
# Plot the world data
ggplot() + 
  geom_path(data = ag_cr_world, aes(x = prob/100, y = importance, color = gov_challenge),
            alpha = 0.3) + 
  geom_point(data = ag_sum_world, aes(x = prob/100, y = importance, 
                                color = gov_challenge), size = 3) +
  geom_text_repel(data = ag_sum_world, aes(x = prob/100, y = importance, 
                                     label = str_wrap(gov_challenge, width = 20)), 
                  point.padding = 0.75, segment.alpha = 0.6) +
  scale_x_continuous(name = "Likelihood of impacting large numbers of people around the world within 10 years", labels = scales::percent, limits = c(0.50, 0.725)) + 
  ylab("Issue importance\n(0 = Not at all important; 3 = Very important)") +
  theme_bw() + theme(legend.position="none") +
  labs(caption = "Source: Center for the Governance of AI")


# ggplot() + 
#   geom_path(data = ag_cr_world, aes(x = prob/100, y = importance, color = gov_challenge),
#             alpha = 0.3) + 
#   geom_point(data = ag_sum_world, aes(x = prob/100, y = importance, 
#                                 color = gov_challenge), size = 3) +
#   geom_text_repel(data = ag_sum_world, aes(x = prob/100, y = importance, 
#                                      label = str_wrap(gov_challenge, width = 20)), 
#                   point.padding = 0.75, segment.alpha = 0.6) +
#   scale_x_continuous(name = "Likelihood of impacting large numbers of people around the world within 10 years", labels = scales::percent) + 
#   ylab("Issue importance\n(0 = Not at all important; 3 = Very important)") +
#   theme_bw() + theme(legend.position="none") +
#   labs(title = "Perceptions of AI governance challenges around the world",
#        caption = "Source: Center for the Governance of AI")
# ggsave("~/Google Drive/AI Public Opinion Surveys/Figures/gov_challenges_world_wide.png",
#        dpi = 300, width = 11, height = 7.5)
```

## Americans who are younger, who have CS or engineering degrees express less concern about AI governance challenges

We performed further analysis by calculating the percentage of respondents in each subgroup who consider each governance challenge to be "very important" for governments and tech companies to manage. (See [Appendix C](#appgovchallenges) for additional data visualizations.) In general, differences in responses are more salient across demographic subgroups than across governance challenges. In a linear multiple regression predicting perceived issue importance using demographic subgroups, governance challenges, and the interaction between the two, we find that the stronger predictors are demographic subgroup variables, including age group and having CS or programming experience. 

Two highly visible patterns emerge from our data visualization. First, a higher percentage of older respondents, compared with younger respondents, consider nearly all AI governance challenges to be "very important." As discussed previously, we find that older Americans, compared with younger Americans, are less supportive of developing AI. Our results here might explain this age gap: older Americans see each AI governance challenge as substantially more important than do younger Americans. Whereas 85% of Americans older than 73 consider each of these issues to be very important, only 40% of Americans younger than 38 do. 

Second, those with CS or engineering degrees, compared with those who do not, rate all AI governance challenges as less important. This result could explain our previous finding that those with CS or engineering degrees tend to exhibit greater support for developing AI. [^numage]

[^numage]: In Table \@ref(tab:aigovregsat), we report the results of a saturated linear model using demographic variables, governance challenges, and the interaction between these two types of variables to predict perceived issue importance. We find that those who are 54-72 or 73 and older, relative to those who are below 38, view the governance issues as more important (two-sided $p$-value < 0.001 for both comparisons). Furthermore, we find that those who have CS or engineering degrees, relative to those who do not, view the governance challenges as less important (two-sided $p$-value < 0.001).

```{r airisksdemo, echo=FALSE, fig.height=10, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.width=7, fig.cap="AI governance challenges: issue importance by demographic subgroups"}

# Helper function to clean up the data 
heatmap_func <- function(dem_var, dem_group) {
  temp <- ag_clean_US %>% group_by_(dem_var, 'gov_challenge') %>% dplyr::summarise(
  p_importance = wtd.mean(importance >= 3, weights = survey_weights),
  importance = wtd.mean(importance, weights = survey_weights),
  group = dem_group
)
  names(temp)[1] <- "characteristic"
  return(as.data.frame(temp))
}

# Summarize data by demographic subgroup and AI governance challenge 
heat_map_res <- rbind(heatmap_func(dem_var = "demo_age", dem_group = "Age group"),
heatmap_func(dem_var = "demo_gender", dem_group = "Gender"),
heatmap_func(dem_var = "demo_white", dem_group = "Race"),
heatmap_func(dem_var = "demo_educ", dem_group = "Education"),
heatmap_func(dem_var = "demo_employ", dem_group = "Employment status"),
heatmap_func(dem_var = "demo_pid3", dem_group = "Political party"),
heatmap_func(dem_var = "demo_income", dem_group = "Household income"),
heatmap_func(dem_var = "demo_rel", dem_group = "Religion"),
heatmap_func(dem_var = "demo_bornagain", dem_group = "Born-again Christian"),
heatmap_func(dem_var = "demo_cs", dem_group = "CS or engineering degree"),
heatmap_func(dem_var = "demo_prog", dem_group = "CS or programming experience"))
# Clean up the data
heat_map_res$characteristic <- factor(heat_map_res$characteristic, 
                                      levels = rev(levels(heat_map_res$characteristic)))
heat_map_res$gov_challenge <- factor(heat_map_res$gov_challenge,
                                     levels =
                                       ag_sum_US$gov_challenge[order(ag_sum_US$importance)])
# Mean center the outcome
heat_map_res$importance_mc <- heat_map_res$importance - 
  wtd.mean(ag_clean_US$importance, weights = ag_clean_US$survey_weights)

# Make first graph
ggplot(heat_map_res, aes(x = gov_challenge, y = characteristic, fill = p_importance))+
  geom_bin2d(color = "white") + xlab("AI governance challenges") + 
  scale_y_discrete(name = "Demographic subgroups",
                   labels = function(x) str_wrap(x, width = 30)) +
  # geom_text(aes(x = gov_challenge, y = characteristic, 
  #               label = roundfunc(p_importance*100, 0)), color = "black", size = 3) + 
    scale_fill_gradient2(name = "Percent who considers the issue very important", 
                         midpoint = mean(heat_map_res$p_importance),
                       low = "#1b7837", mid = "#f7f7f7", high = "#762a83", 
                       labels = scales::percent) + 
  # scale_fill_gradient(name = "Percent who considers the issue very important",
  #                     low = "white", high = "#00003f", labels = scales::percent) + 
    theme_bw() + theme(legend.position = "bottom",
                       axis.text.x = element_text(angle = 270, hjust = 0)) +
  guides(fill = guide_colourbar(title.position="top", title.hjust = 0.5,
                                barwidth = 15)) +
  labs(
       caption = "Source: Center for the Governance of AI")

```

```{r airisksdemoreg, echo=FALSE, fig.height=10, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE, dev='pdf', dpi=300, fig.width=7, fig.cap="AI governance challenges: issue importance by demographic subgroups"}

# Regression code commented out because it takes a long time to run

# ai_importance_gov <- lm_robust(formula = as.formula("importance ~ gov_challenge"), 
#      data = ag_clean_US, weights = ag_clean_US$survey_weights, 
#      clusters = ag_clean_US$respondent_id)
# ai_imp_regs_gov <- data.frame(coef = ai_importance_gov$coefficients, 
#                           se = ai_importance_gov$std.error,
#                           p_value = ai_importance_gov$p.value)
# ai_imp_regs_gov$stars <- ""
# ai_imp_regs_gov$stars[ai_imp_regs_gov$p_value < 0.05] <- "*"
# ai_imp_regs_gov$stars[ai_imp_regs_gov$p_value < 0.01] <- "**"
# ai_imp_regs_gov$stars[ai_imp_regs_gov$p_value < 0.001] <- "***"
# ai_imp_regc <- data.frame(
#   varnames = gsub("gov_challenge",
#                          "", row.names(ai_imp_regs_gov)),
#   coef = paste0(roundfunc(ai_imp_regs_gov[,1], 2), " (", roundfunc(ai_imp_regs_gov[,2], 2), ")",
#        ai_imp_regs_gov$stars))
# 
# ai_importance_md <- lm_robust(formula = as.formula("importance ~ demo_age + demo_gender + demo_white + demo_educ +
#      demo_employ + demo_income + demo_pid3 + demo_rel + demo_bornagain +
#      demo_cs + demo_prog + gov_challenge + demo_age:gov_challenge + demo_gender:gov_challenge +
#        demo_white:gov_challenge + demo_educ:gov_challenge + demo_employ:gov_challenge +
#        demo_income:gov_challenge + demo_pid3:gov_challenge +
#        demo_rel:gov_challenge + demo_bornagain:gov_challenge +
#        demo_cs:gov_challenge + demo_prog:gov_challenge"), 
#      data = ag_clean_US, weights = ag_clean_US$survey_weights, 
#      clusters = ag_clean_US$respondent_id)
# save(ai_importance_md, file = "~/Google Drive/AI Public Opinion Surveys/R/yougov/ai_importance_md.RData")

# Load in the regression output
load(paste0(wd, "ai_importance_md.RData"))
ai_imp_regs <- data.frame(coef = ai_importance_md$coefficients[1:32], 
                          se = ai_importance_md$std.error[1:32],
                          p_value = ai_importance_md$p.value[1:32])
ai_imp_regs$stars <- ""
ai_imp_regs$stars[ai_imp_regs$p_value < 0.05] <- "*"
ai_imp_regs$stars[ai_imp_regs$p_value < 0.01] <- "**"
ai_imp_regs$stars[ai_imp_regs$p_value < 0.001] <- "***"

f_pval <- roundfunc(pf(ai_importance_md$fstatistic[1], 
                ai_importance_md$fstatistic[2], 
                ai_importance_md$fstatistic[3], lower.tail=F), 4)

ai_imp_regc <- rbind(data.frame(
  varnames = gsub(paste0(paste0(demo_var, collapse = "|"), "|gov_challenge"), 
                         "", row.names(ai_imp_regs)),
  coef = paste0(roundfunc(ai_imp_regs[,1], 2), " (", roundfunc(ai_imp_regs[,2], 2), ")",
       ai_imp_regs$stars)),
  data.frame(varnames = "\\textit{N} = 10000 observations, 2000 respondents",
             coef = paste0("\\textit{F}(", round(ai_importance_md$fstatistic[2]), ",",
                           round(ai_importance_md$fstatistic[3]), ") = ",
                           roundfunc(ai_importance_md$fstatistic[1]),
             "; \\textit{p}-value: <0.001")))

```

## Americans place the most trust in the U.S. military and universities to build AI; trust tech companies and non-governmental organizations more than the government to manage the technology 

Respondents were asked how much confidence they have in various actors to develop AI. They were randomly assigned five actors out of 15 to evaluate. We provided a short description of actors that are not well-known to the public (e.g., NATO, CERN, and OpenAI).

Also, respondents were asked how much confidence, if any, they have in various actors to manage the development and use of AI in the best interests of the public. They were randomly assigned five out of 15 actors to evaluate. Again, we provided a short description of actors that are not well-known to the public (e.g., AAAI and Partnership on AI). Confidence was measured using the same four-point scale described above. [^actorsnote] 

[^actorsnote]: The two sets of 15 actors differed slightly because for some actors it seemed inappropriate to ask one or the other question. See [Appendix B](#trustdevai) for the exact wording of the questions and descriptions of the actors. 

Americans do not express great confidence in most actors to develop or to manage AI, as reported in Figures \@ref(fig:trustdevtable1) and \@ref(fig:trustdevtable2). A majority of Americans do not have a "great deal" or even a "fair amount" of confidence in any institution, except university researchers, to develop AI. Furthermore, Americans place greater trust in tech companies and non-governmental organizations (e.g., OpenAI) than in governments to manage the development and use of the technology. 

University researchers and the U.S. military are the most trusted groups to develop AI: about half of Americans express a "great deal" or even a "fair amount" of confidence in them. Americans express slightly less confidence in tech companies, non-profit organizations (e.g., OpenAI), and American intelligence organizations. Nevertheless, opinions toward individual actors within each of these groups vary. For example, while 44% of Americans indicated they feel a "great deal" or even a "fair amount" of confidence in tech companies, they rate Facebook as the least trustworthy of all the actors. More than four in 10 indicate that they have no confidence in the company. [^fbfn]

[^fbfn]: Our survey was conducted between June 6 and 14, 2018, shortly after the fallout of the Facebook/Cambridge Analytica scandal. On April 10-11, 2018, Facebook CEO Mark Zuckerberg testified before the U.S. Congress regarding the Cambridge Analytica data leak. On May 2, 2018, Cambridge Analytica announced its shutdown. Nevertheless, Americans' distrust of the company existed before the Facebook/Cambridge Analytica scandal. In a pilot survey that we conducted on Mechanical Turk during July 13-14, 2017, respondents indicated a substantially lower level of confidence in Facebook, compared with other actors, to develop and regulate AI.

The results on the public's trust of various actors to manage the develop and use of AI provided are similar to the results discussed above. Again, a majority of Americans do not have a "great deal" or even a "fair amount" of confidence in any institution to manage AI. In general, the public expresses greater confidence in non-governmental organizations than in governmental ones. Indeed, 41% of Americans express a "great deal" or even a "fair amount" of confidence in "tech companies," compared with 26% who feel that way about the U.S. federal government. But when presented with individual big tech companies, Americans indicate less trust in each than in the broader category of "tech companies." Once again, Facebook stands out as an outlier: respondents give it a much lower rating than any other actor. Besides "tech companies," the public places relatively high trust in intergovernmental research organizations (e.g., CERN), the Partnership on AI, and non-governmental scientific organizations (e.g., AAAI). Nevertheless, because the public is less familiar with these organizations, about one in five respondents give a "don't know" response. 

Mirroring our findings, recent survey research suggests that while Americans feel that AI should be regulated, they are unsure _who_ the regulators should be. When asked who "should decide how AI systems are designed and deployed," half of Americans indicated they do not know or refused to answer [@west2018divided]. Our survey results seem to reflect Americans' general attitudes toward public institutions. According to a 2016 Pew Research Center survey, an overwhelming majority of Americans have "a great deal" or "a fair amount" of confidence in the U.S. military and scientists to act in the best interest of the public. In contrast, public confidence in elected officials is much lower: 73% indicated that they have "not too much" or "no confidence" [@funk2017]. Less than one-third of Americans thought that tech companies do what's right "most of the time" or "just about always"; moreover, more than half think that tech companies have too much power and influence in the U.S. economy [@smith2018]. Nevertheless, Americans' attitude toward tech companies is not monolithic but varies by company. For instance, our research findings reflect the results from a 2018 survey, which reported that a higher percentage of Americans trusted Apple, Google, Amazon, Microsoft, and Yahoo to protect user information than trust Facebook to do so [@ipsosreuters2018].

```{r trustdevtable1, dev='pdf', dpi=300, fig.width=7, fig.cap="Trust in various actors to develop AI: distribution of responses", echo=FALSE, fig.height=8, fig.keep='all', cache=TRUE, warning=FALSE}

rm("ag_clean_all")
rm("ag_clean_US")
rm("ag_clean_world")
rm("ag_cr_all")
rm("ag_cr_US")
rm("ag_cr_world")

# Labels for the actors
dev_actors <- c("U.S. military", "U.S. civilian government",
                "NSA", "FBI", "CIA", "NATO", 
                "Intergovernmental research organizations (e.g., CERN)",
                "Tech companies", "Google", "Facebook",
                "Apple", "Microsoft", "Amazon", "Non-profit (e.g., OpenAI)", 
                "University researchers")
# Overall mean
ai_dev_overall_mean <- wtd.mean(relabel_var(as.numeric(unlist(d[,paste0("Q6_", 1:15)])), 
            c(1:5, 8, 9), c(3, 2, 1, 0, NA, NA, NA)), 
         weights = rep(d$survey_weights, 15))
# Helper function
ai_dev_func <- function(variable_number, output_type) {
  catvar_func(outcome = dev_actors[variable_number], 
              outcome_var = d[,paste0("Q6_", variable_number)], 
              shown = d0[,paste0("Q6_org_", variable_number)] == 1,
              label_var = d0[,paste0("Q6_", variable_number)], 
              num_missing = 8, num_DK = 5,
              new_values = c(3, 2, 1, 0, NA, NA, NA),
              survey_weights = d$survey_weights,
              missing_recode = ai_dev_overall_mean,
              output_type)
}

# Frequency table
trust_dev_table <- do.call(rbind, lapply(X = 1:15, ai_dev_func, 
                                   output_type = "value_table"))
# Summary statistics
trust_dev_res <- do.call(rbind, lapply(X = 1:15, ai_dev_func, 
                                   output_type = "value_sum"))

# Classify the actors
# Summary statistics
classify_org <- data.frame(outcome = unique(trust_dev_res$outcome),
                           Org = c(rep("U.S. government", 5),
                                   rep("International", 2),
                                   rep("Corporate", 6),
                                   rep("Other", 2)))
trust_dev_res <- merge(x = trust_dev_res, y = classify_org, all.x = TRUE)
trust_dev_res$Org <- factor(trust_dev_res$Org, 
                            levels = c("U.S. government", "International",
                                           "Corporate", "Other"))
trust_dev_res$group <- "Trust in various actors to develop AI in the interest of the public"
# Frequency table
trust_dev_table <- merge(x = trust_dev_table, y = classify_org, all.x = TRUE)
trust_dev_table$Org <- factor(trust_dev_table$Org, 
                            levels = c("U.S. government", "International",
                                           "Corporate", "Other"))
trust_dev_table$group <- "Trust in various actors to develop AI in the interest of the public"


# Labels for the actors
manage_actors <- c("U.S. federal government", "U.S. state governments", 
                "International organizations", "UN", 
                "Intergovernmental research organizations (e.g., CERN)", 
                "Tech companies", "Google",
                "Facebook", "Apple", "Microsoft", "Amazon", 
                "Non-government scientific organization (e.g., AAAI)", "Partnership on AI")
# Overall mean
ai_manage_overall_mean <- wtd.mean(relabel_var(as.numeric(unlist(d[,paste0("Q7_", 1:13)])), 
            c(1:5, 8, 9), c(3, 2, 1, 0, NA, NA, NA)), 
         weights = rep(d$survey_weights, 13))
# Helper function
ai_manage_func <- function(variable_number, output_type) {
  catvar_func(outcome = manage_actors[variable_number], 
              outcome_var = d[,paste0("Q7_", variable_number)], 
              shown = d[,paste0("Q7_org_", variable_number)] == 1,
              label_var = d0[,paste0("Q7_", variable_number)], 
              num_missing = 8, num_DK = 5,
              new_values = c(3, 2, 1, 0, NA, NA, NA), 
              missing_recode = ai_manage_overall_mean,
              output_type)
}

# Frequency table
trust_manage_table <- do.call(rbind, lapply(X = 1:13, ai_manage_func, 
                                   output_type = "value_table"))
# Summary statistics
trust_manage_res <- do.call(rbind, lapply(X = 1:13, ai_manage_func, 
                                   output_type = "value_sum"))

# Classify the actors
# Summary statistics
classify_org <- data.frame(outcome = unique(trust_manage_res$outcome),
                           Org = c(rep("U.S. government", 2),
                                   rep("International", 3),
                                   rep("Corporate", 6),
                                   rep("Other", 2)))
trust_manage_res <- merge(x = trust_manage_res, y = classify_org, all.x = TRUE)
trust_manage_res$Org <- factor(trust_manage_res$Org, 
                            levels = rev(c("Other", "Corporate", 
                                       "International", "U.S. government")))
trust_manage_res$group <- "Trust in various actors to manage AI in the interest of the public"
trust_res <- rbind(trust_dev_res, trust_manage_res)

# Frequency table
trust_manage_table <- merge(x = trust_manage_table, y = classify_org, all.x = TRUE)
trust_manage_table$Org <- factor(trust_manage_table$Org, 
                            levels = rev(c("Other", "Corporate", 
                                       "International", "U.S. government")))
trust_manage_table$group <- "Trust in various actors to manage AI in the interest of the public"

# Clean up the frequency table
trust_table <- rbind(trust_dev_table, trust_manage_table)
trust_table$labels[trust_table$labels %in% c("I don't know", "Skipped")] <- 
  "Don't know/Skipped"
trust_table <- trust_table %>% group_by(outcome, labels, group, new_values, Org) %>%
  dplyr::summarise(Prop = sum(Prop))

# Change the factors
trust_table$labels <- factor(trust_table$labels,
     levels = rev(c("3. A great deal of confidence", "2. A fair amount of confidence", 
                    "1. Not too much confidence", "0. No confidence",
                    "Don't know/Skipped")))
trust_table$percent <- trust_table$Prop*100
trust_table$percent_zero <- trust_table$percent == 0 
trust_table$percent <- ifelse(round(trust_table$percent) == 0, "<1", round(trust_table$percent))
trust_table$percent[trust_table$percent_zero] <- 0

# Make the graph: develop AI
ggplot(data = trust_table[trust_table$group == unique(trust_table$group)[1],],
       aes(x=outcome, y=Prop, fill=labels)) +
  geom_bar(stat="identity", position = "fill", alpha = 0.6) +
  geom_text(aes(label = percent),
            position = position_stack(vjust = 0.5), size = 3) +
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
                         name = "Actors") + 
  scale_y_continuous(name = "Percentage of respondents", 
                     labels = scales::percent,
                     limits = c(0, 1), expand = c(0, 0)) +
  coord_flip() +
  theme_bw() + theme(legend.position = "bottom") +
  facet_grid(Org~., scales = "free_y", space = "free_y",
             labeller = label_wrap_gen(width = 35)) +
  scale_fill_manual(values = c("grey65", "#f2f0f7", "#cbc9e2", "#9e9ac8", "#6a51a3"), 
                    name = "Responses") +
   guides(fill = guide_legend(reverse = TRUE, ncol = 2)) +
  labs(caption = "Source: Center for the Governance of AI")

```


```{r trustdevtable2, dev='pdf', dpi=300, fig.width=7, fig.cap="Trust in various actors to manage AI: distribution of responses", echo=FALSE, fig.height=8, fig.keep='all', cache=TRUE, warning=FALSE}

# Make the graph: develop AI
ggplot(data = trust_table[trust_table$group == unique(trust_table$group)[2],],
       aes(x=outcome, y=Prop, fill=labels)) +
  geom_bar(stat="identity", position = "fill", alpha = 0.6) +
  geom_text(aes(label = percent),
            position = position_stack(vjust = 0.5), size = 3) +
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
                         name = "Actors") + 
  scale_y_continuous(name = "Percentage of respondents", 
                     labels = scales::percent,
                     limits = c(0, 1), expand = c(0, 0)) +
  coord_flip() +
  theme_bw() + theme(legend.position = "bottom") +
  facet_grid(Org~., scales = "free_y", space = "free_y",
             labeller = label_wrap_gen(width = 10)) +
  scale_fill_manual(values = c("grey65", "#f2f0f7", "#cbc9e2", "#9e9ac8", "#6a51a3"), 
                    name = "Responses") +
   guides(fill = guide_legend(reverse = TRUE, ncol = 2)) +
  labs(caption = "Source: Center for the Governance of AI")

```


```{r trustcoef, dev='pdf', dpi=300, fig.width=6, fig.cap="Trust in various actors to develop and manage AI in the interest of the public", echo=FALSE, fig.height=9, fig.keep='all', cache=TRUE, warning=FALSE}

# Clean the outcome factors
trust_res$outcome <- factor(trust_res$outcome, levels = rev(levels(trust_res$outcome)))

# Grid lines
grid_d <- rbind(data.frame(Org = "U.S. government", 
                           grid = seq(1.5, 6.5, by = 1)),
                data.frame(Org = "International", 
                           grid = seq(1.5, 3.5, by = 1)),
                data.frame(Org = "Corporate", 
                           grid = seq(1.5, 5.5, by = 1)),
                data.frame(Org = "Other", 
                           grid = seq(1.5, 3.5, by = 1)))
grid_d$Org <- factor(grid_d$Org, levels = levels(trust_res$Org))

# Plot the graph
ggplot(data = trust_res, aes(x = outcome, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_pointrange(aes(color = group, shape = group), 
                  position = position_dodge(width = 1)) + 
  geom_vline(data = grid_d, aes(xintercept = grid), alpha = 0.2) + 
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
                          name = "Actors") + 
  scale_y_continuous(
    name = "Perceived trust (0 = No confidence at all;\n3 = A great deal of confidence)") + theme_bw() + 
  facet_grid(Org~., scales = "free_y", space = "free_y", 
             labeller = label_wrap_gen(width = 35)) +
  theme(legend.position="bottom", legend.direction = "vertical",
        panel.grid.major.y = element_blank()) +
  scale_color_manual(values = c("#92c5de", "coral"), name = "Outcome measures") +
  scale_shape_discrete(name = "Outcome measures") + 
  labs(
       caption = "Source: Center for the Governance of AI")

# ggplot(data = trust_res, aes(x = outcome, y = num, 
#                                  ymin = qnorm(0.025) * se + num,
#                                  ymax = qnorm(0.975) * se + num)) +
#   geom_pointrange(aes(color = group, shape = group), 
#                   position = position_dodge(width = 1)) + 
#   geom_vline(data = grid_d, aes(xintercept = grid), alpha = 0.2) + 
#   coord_flip() + 
#   scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
#                           name = "Actors") + 
#   scale_y_continuous(
#     name = "Perceived trust (0 = No confidence at all;\n3 = A great deal of confidence)") + theme_bw() + 
#   facet_grid(Org~., scales = "free_y", space = "free_y", 
#              labeller = label_wrap_gen(width = 35)) +
#   theme(legend.position="bottom", legend.direction = "horizontal",
#         panel.grid.major.y = element_blank()) +
#   scale_color_manual(values = c("#92c5de", "coral"), name = "Outcome measures") +
#   scale_shape_discrete(name = "Outcome measures") + 
#   labs(title = "Perceived trust in various actors to develop and manage AI",
#        caption = "Source: Center for the Governance of AI")
# 
# ggsave(filename = "~/Google Drive/AI Public Opinion Surveys/Figures/wide_trust_figure.png",
#        dpi = 300, width = 11, height = 7.5)
```

```{r trusttab, dev='pdf', dpi=300, fig.width=6, echo=FALSE, fig.height=9, fig.keep='all', cache=TRUE, warning=FALSE}
# Make a table
# Clean the data
trust_dev_res_clean <- trust_dev_res[,c("outcome", "sum_stat", "Org")]
trust_dev_res_clean$outcome <- as.character(trust_dev_res_clean$outcome)
trust_dev_res_clean$Org <- as.character(trust_dev_res_clean$Org)
names(trust_dev_res_clean)[names(trust_dev_res_clean) == "sum_stat"] <- "dev_ai"
trust_manage_res_clean <- trust_manage_res[,c("outcome", "sum_stat", "Org")]
trust_manage_res_clean$outcome <- as.character(trust_manage_res_clean$outcome)
trust_manage_res_clean$Org <- as.character(trust_manage_res_clean$Org)
names(trust_manage_res_clean)[names(trust_manage_res_clean) == "sum_stat"] <- "manage_ai"
# Marge the data
trust_table <- merge(x = trust_dev_res_clean, y = trust_manage_res_clean, 
                     all = TRUE, by = "outcome")
trust_table$Org <- ifelse(is.na(trust_table$Org.x), trust_table$Org.y, trust_table$Org.x)
trust_table <- trust_table[,c("outcome", "Org", "dev_ai", "manage_ai")]
trust_table$dev_ai <- as.character(trust_table$dev_ai)
trust_table$manage_ai <- as.character(trust_table$manage_ai)

trust_table$dev_ai[is.na(trust_table$dev_ai)] <- ""
trust_table$manage_ai[is.na(trust_table$manage_ai)] <- ""
trust_table$Org <- factor(trust_table$Org, 
                          levels = c("U.S. government", "International", 
                                     "Corporate", "Other"))
trust_table$outcome <- factor(trust_table$outcome, levels = rev(levels(trust_res$outcome)))
trust_table <- trust_table[order(trust_table$Org, trust_table$outcome),]
# Remove the extra info
trust_table$dev_ai <- gsub("Mean: ", "", trust_table$dev_ai)
trust_table$manage_ai <- gsub("Mean: ", "", trust_table$manage_ai)

```

\newpage

# AI policy and U.S.-China relations


## Americans underestimate the U.S. and China's AI research and development 

In this survey experiment, we asked respondents to consider either the U.S. or China's status in AI research and development (R&D). (Details of the survey experiment are found in [Appendix B](#airesearchcompare).) Respondents were asked the following:

>Compared with other industrialized countries, how would you rate [the U.S./China] in AI research and development?

By almost any metric of absolute achievement (not per-capita achievement), the U.S. and China are the world leaders in the research and development of AI. The U.S. and China led participation in the 2017 AAAI Conference, one of the important ones in the field of AI research; 34% of those who presented papers had a U.S. affiliation while 23% had a Chinese affiliation [@goldfarb2018ai]. The U.S. and China also have the highest percentage of the world's AI companies, 42% and 23%, respectively [@chinausreport2017]. Most clearly, the U.S. and China have the largest technology companies focused on developing and using AI (Google, Facebook, and Amazon in the U.S.; Tencent, Alibaba, and Baidu in China). 

Yet, only a minority of the American public thinks the U.S. or China's AI R&D is the "best in the world," as reported in Figure \@ref(fig:uschina). Our survey result seems to reflect the gap between experts and the public's perceptions of U.S.'s scientific achievements in general. While 45% of scientists in the American Association for the Advancement of Science think that scientific achievements in the U.S. are the best in the world, only 15% of the American public express the same opinion [@funk2015].


According to our survey, there is not a clear perception by Americans that the U.S. has the best AI R&D in the world. While 10% of Americans believe that the U.S. has the best AI R&D in the world, 7% think that China does. Still, 36% of Americans believe that the U.S.'s AI R&D is "above average" while 45% think China's is "above average." Combining these into a single measure of whether the country has "above average" or "best in the world" AI R&D, Americans do not perceive the U.S. to be superior, and the results lean towards the perception that China is superior. Note that we did not ask for a direct comparison, but instead asked each respondent to evaluate one country independently on an absolute scale [Appendix C](#appuschinacomp). 

Our results mirror those from a recent survey that finds that Americans think that China's AI capability will be on par with the U.S.'s in 10 years [@west2018worries]. The American public's perceptions could be caused by media narratives that China is catching up to the U.S. in AI capability [@kai2018ai]. Nevertheless, another study suggests that although China has greater access to big data than the U.S., China's AI capability is about half of the U.S.'s [@ding2018]. Exaggerating China's AI capability could exacerbate growing tensions between the U.S. and China [@zwetsloot2018]. As such, future research should explore how factual -- non-exaggerated --  information about American and Chinese AI capabilities influences public opinions.

```{r uschina, echo=FALSE, fig.height=7, fig.keep='all', fig.cap="Comparing Americans' perceptions of U.S. and China's AI research and development quality", cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf', fig.width=7}

# Frequency table
# Overall mean for recoding missing values
rd_overall_mean <- wtd.mean(relabel_var(ifelse(is.na(d$Q12a), d$Q12b, d$Q12a),
                                        c(1:5, 8, 9), c(3, 2, 1, 0, NA, NA, NA)),
                            weights = d$survey_weights, na.rm = TRUE)
# U.S. 
us_rd_value_table <- catvar_func(
  outcome = label(d0$Q12a),
  outcome_var = d$Q12a,
  label_var = d0$Q12a,
  output_type = "value_table",
  shown = d$q12a_treat == 1,
  num_missing = 8,
  num_DK = 5,
  new_values = c(3, 2, 1, 0, NA, NA, NA),
  missing_recode = rd_overall_mean
  )  
us_rd_value_table$num <- c(3, 2, 1, 0, 8, 9)
us_rd_value_table$country <- "U.S."
# China
china_rd_value_table <- catvar_func(
  outcome = label(d0$Q12b),
  outcome_var = d$Q12b,
  label_var = d0$Q12b,
  output_type = "value_table",
  shown = d$q12a_treat == 2,
  num_missing = 8,
  num_DK = 5,
  new_values = c(3, 2, 1, 0, NA, NA, NA),
  missing_recode = rd_overall_mean
  )  
china_rd_value_table$num <- c(3, 2, 1, 0, 8, 9)
china_rd_value_table$country <- "China"
# Combine the data
rd_value_table <- rbind(us_rd_value_table, china_rd_value_table)

# Numerical values
# U.S.
us_rd_value_sum <- catvar_func(
  outcome = label(d0$Q12a),
  outcome_var = d$Q12a,
  label_var = d0$Q12a,
  output_type = "num_value",
  shown = d$q12a_treat == 1,
  num_missing = 8,
  num_DK = 5,
  new_values = c(3, 2, 1, 0, NA, NA, NA),
  missing_recode = rd_overall_mean
  )  
us_rd_value_sum$country <- "U.S."
# China
china_rd_value_sum <- catvar_func(
  outcome = label(d0$Q12b),
  outcome_var = d$Q12b,
  label_var = d0$Q12b,
  output_type = "num_value",
  shown = d$q12a_treat == 2,
  num_missing = 8,
  num_DK = 5,
  new_values = c(3, 2, 1, 0, NA, NA, NA),
  missing_recode = rd_overall_mean
  )  
china_rd_value_sum$country <- "China"
# combine the num_value datasets
rd_value_sum <- rbind(us_rd_value_sum, china_rd_value_sum)
# Set factor levels
rd_value_table$country <- factor(rd_value_table$country, levels = c("U.S.", "China"))
rd_value_sum$country <- factor(rd_value_sum$country, levels = c("U.S.", "China"))

# Make the graph
rd_value_table$labels <- factor(rd_value_table$labels, 
                                levels = rd_value_table$labels[c(4:1, 5:6)])
ggplot() +
  geom_bar(data = rd_value_table[rd_value_table$Prop !=0,], 
           aes(x = num, y = Prop, fill = labels), stat = "identity",
           alpha = 0.6, color = "black") +
    geom_errorbar(data = 
                  rd_value_table[rd_value_table$Prop !=0,], 
                aes(x = num, ymin = Prop + qnorm(0.025)*se,
                    ymax = Prop + qnorm(0.975)*se), width = 0.1) +
  geom_text(data = rd_value_table, aes(x = num, 
                                           label = roundfunc(Prop*100, 0)), 
            y = 0.02, nudge_x = 0.25) +
  scale_x_continuous(breaks = rd_value_table$num[order(rd_value_table$num)],
    labels = str_wrap(rd_value_table$labels[order(rd_value_table$num)], 
                      width = 15)) +
  facet_grid(country~group, scales = "free_x", space = "free_x") + theme_bw() +
  geom_text(data = rd_value_sum, aes(x = 1.5, label = sum_stat,
                                       y = max(rd_value_table$Prop)+0.1)) +
  scale_y_continuous(labels = scales::percent, 
                     limits = c(0, max(rd_value_table$Prop)+0.1)) +
  xlab("Responses") + ylab("Percentage of respondents") + 
  scale_fill_manual(values = c("#f2f0f7", "#cbc9e2", "#9e9ac8", "#6a51a3", "grey65")) +
  labs(
       caption = "Source: Center for the Governance of AI") + 
  theme(legend.position = "none")

```

## Communicating the dangers of a U.S.-China arms race requires explaining policy trade-offs {#subsecexperimentchina}

In this survey experiment, respondents were randomly assigned to consider different arguments about a U.S.-China arms race. (Details of the survey experiment are found in [Appendix B](#armsraceexp).) All respondents were given the following prompt:

>Leading analysts believe that an AI arms race is beginning, in which the U.S. and China are investing billions of dollars to develop powerful AI systems for surveillance, autonomous weapons, cyber operations, propaganda, and command and control systems.

Those in the treatment condition were told they would read a short news article. The three treatments were:

1. **Pro-nationalist treatment**: The U.S. should invest heavily in AI to stay ahead of China; quote from a senior National Security Council official

2. **Risks of arms race treatment**: The U.S.-China arms race could increase the risk of a catastrophic war; quote from Elon Musk

3. **One common humanity treatment**: The U.S.-China arms race could increase the risk of a catastrophic war; quote from Stephen Hawking about using AI for the good of all people rather than destroying civilization 

Respondents were asked to consider two statements and indicate whether they agree or disagree with them:

- The U.S. should invest more in AI military capabilities to make sure it doesn't fall behind China's, even if doing so may exacerbate the AI arms race.

- The U.S. should work hard to cooperate with China to avoid the dangers of an AI arms race, even if doing so requires giving up some of the U.S.'s advantages. Cooperation could include collaborations between American and Chinese AI research labs, or the U.S. and China creating and committing to common safety standards for AI.

Americans, in general, weakly agree that the U.S. should invest more in AI military capabilities _and_ cooperate with China to avoid the dangers of an AI arms race, as seen in Figure \@ref(fig:armsrace). Many respondents do not think that the two policies are mutually exclusive. The correlation between responses to the two statements, unconditional on treatment assignment, is only -0.05. In fact, 29% of those who agree that the U.S. and China should cooperate also agree that the U.S. should invest more in AI military capabilities. (See Figure \@ref(fig:armsracecorrfig) for the conditional percentages.)

Respondents assigned to read about the risks of an arms race (Treatment 2) indicate significantly higher agreement with the pro-cooperation statement (Statement 2) than the investing in AI military capabilities statement (Statement 1), according to Figure \@ref(fig:armsracediff). Those assigned to Treatment 2 are more likely to view the two statements as mutually exclusive. In contrast, respondents assigned to the other conditions indicate similar levels of agreement with both statements.

After estimating the treatment effects, we find that the experimental messages do little to change the respondents' preferences. Treatment 2 is the one exception. Treatment 2 decreases respondents' agreement with the statement that the U.S. should invest more in AI military capabilities by 27%, as seen in Figure \@ref(fig:armsraceregression). Future research could focus on testing more effective messages, such as op-eds [@coppock2018long] or videos [@paluck2015does], which explains that U.S.'s investment in AI for military use will decrease the likelihood of cooperation with China.

```{r armsrace, echo=FALSE, fig.height=4.5, fig.keep='all', warning=FALSE, cache=TRUE, fig.cap = "Responses from U.S.-China arms race survey experiment", dpi = 300, dev = 'pdf', fig.width=7}

# Overall means for the two outcomes (to recode the missing values)
Q12_overall_mean <- wtd.mean(relabel_var(d$Q12, c(1:6, 8, 9),
                                         c(2, 1, 0, -1, -2, NA, NA, NA)),
                             weights = d$survey_weights, na.rm = TRUE)
Q13_overall_mean <- wtd.mean(relabel_var(d$Q13, c(1:6, 8, 9),
                                         c(2, 1, 0, -1, -2, NA, NA, NA)),
                             weights = d$survey_weights, na.rm = TRUE)

# Function to generate the results
china_exp <- function(varname, outcome, exp_group, output_type = "num_value",
                      missing_recode,
                      new_values = c(2, 1, 0, -1, -2, NA, NA, NA)) {
  return(data.frame(catvar_func(
  outcome = outcome,
  outcome_var = d[,varname],
  label_var = d0[,varname],
  output_type = output_type,
  shown = (d$q12_treat == exp_group),
  num_missing = 8,
  num_DK = 6,
  new_values = new_values,
  missing_recode = missing_recode
  ), exp_group = exp_group))  
}
# Statement 1
exp_invest <- lapply(1:4, china_exp, 
                           varname = "Q12", missing_recode = Q12_overall_mean,
                           new_values = c(2, 1, 0, -1, -2, NA, NA, NA),
                     outcome = "Agreement with statement that U.S. should invest more in AI military capabilities",
       output_type = "num_value") %>% do.call(what = rbind)
# Statement 2
exp_cooperate <- lapply(1:4, china_exp, 
                           varname = "Q13", missing_recode = Q13_overall_mean,
                           new_values = c(2, 1, 0, -1, -2, NA, NA, NA),
                     outcome = "Agreement with statement that U.S. should work hard to cooperate with China to avoid dangers of AI arms race",
       output_type = "num_value") %>% do.call(what = rbind)
# Clean up the data
ar_groups <- c("Control", "Treatment 1: Pro-nationalist","Treatment 2: Risks of arms race", 
  "Treatment 3: One common humanity")
exp_l <- data.frame(exp_group = 1:4, exp_group_l = ar_groups)
exp_outcome <- merge(x = rbind(exp_invest, exp_cooperate), y = exp_l, all.x = TRUE)
exp_outcome$sum_stat <- gsub(pattern = "; ", replacement = ";\n", exp_outcome$sum_stat)
exp_outcome$exp_group_l <- factor(x = exp_outcome$exp_group_l, 
                                  levels = rev(levels(exp_outcome$exp_group_l)))
exp_outcome$sum_stat_s <- 
  gsub(pattern = "Mean: ", replacement = "", exp_outcome$sum_stat)
# Make the plot
ggplot(data = exp_outcome, aes(x = exp_group_l, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_pointrange(position = position_dodge(width = 0.9)) + 
  geom_text(aes(label = sum_stat_s), nudge_x = 0.4, alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 20),
                         name = "Experimental groups") +
  expand_limits(x = c(1, 5)) +
  scale_y_continuous(
    name = "Agreement/disagreement with statement\n(-2 = Strongly disagree; 2 = Strongly agree)") + 
  labs(
       source = "Governance of AI Program") +
  facet_grid(~outcome, labeller = label_wrap_gen(width = 45)) + theme_bw()

```


```{r armsraceregression, echo=FALSE, fig.height=3.5, fig.keep='all', cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Effect estimates from U.S.-China arms race survey experiment"}
# Regression results
# Clean up the data
d$q12_treat_clean <- relabel_var(d$q12_treat, c(1:4, 8, 9),
                                 c(ar_groups, NA, NA))
# Statement 1
d$Q12_clean <-
  relabel_var(
    old_var = d$Q12,
    old_labels = c(1:6, 8, 9),
    new_labels = c(2, 1, 0,-1,-2, NA, NA, NA)
  )
d$Q12_missing <- is.na(d$Q12_clean)
d$Q12_clean[is.na(d$Q12_clean)] <- Q12_overall_mean
# Attrition check
# Attrition rates by experimental groups
q12_attrition <-
  d %>% group_by(q12_treat_clean) %>% dplyr::summarise(attrition_prop = mean(Q12_missing) *
                                                         100,
                                                       DK_prop = mean(Q12 == 6) * 100)
q12_attrition$missing_prop <-
  q12_attrition$attrition_prop - q12_attrition$DK_prop

# Statement 2
d$Q13_clean <-
  relabel_var(
    old_var = d$Q13,
    old_labels = c(1:6, 8, 9),
    new_labels = c(2, 1, 0,-1,-2, NA, NA, NA)
  )
d$Q13_missing <- is.na(d$Q13_clean)
d$Q13_clean[is.na(d$Q13_clean)] <- Q13_overall_mean
d$q13_treat_clean <- d$q12_treat_clean
# Attrition rates by experimental groups
q13_attrition <-
  d %>% group_by(q13_treat_clean) %>% dplyr::summarise(attrition_prop = mean(Q13_missing) *
                                                         100,
                                                       DK_prop = mean(Q13 == 6) * 100)
q13_attrition$missing_prop <-
  q13_attrition$attrition_prop - q13_attrition$DK_prop


# Linear regression function
survey_experiment_func <-
  function(outcome_var, exp_group, outcome) {
    survey_weights <-
      d$survey_weights[d$q12_treat_clean %in% c("Control", exp_group)]
    if (outcome_var == "Q12_clean") {
      d$q12_treatment <- d$q12_treat_clean == exp_group
      if (sum(q12_attrition$attrition_prop[q12_attrition$q12_treat_clean %in%
                                           c("Control", exp_group)] > 0.1) > 0) {
        my_formula <-
          as.formula("Q12_clean ~ q12_treatment + scale(Q12_missing) + q12_treatment:scale(Q12_missing)")
      } else {
        my_formula <- as.formula("Q12_clean ~ q12_treatment")
      }
      md <- lm(my_formula,
                       data = d[d$q12_treat_clean %in% c("Control", exp_group), ],
                       weights = survey_weights)
      md <- coeftest(md, vcov = vcovHC(md, type="HC2"))
    } else {
      d$q13_treatment <- d$q13_treat_clean == exp_group
      if (sum(q13_attrition$attrition_prop[q13_attrition$q13_treat_clean %in%
                                           c("Control", exp_group)] > 0.1) > 0) {
        my_formula <-
          as.formula("Q13_clean ~ q13_treatment + scale(Q13_missing) + q13_treatment:scale(Q13_missing)")
      } else {
        my_formula <- as.formula("Q13_clean ~ q13_treatment")
      }
      md <- lm(my_formula,
                       data = d[d$q13_treat_clean %in% c("Control", exp_group), ],
                       weights = survey_weights)
      md <- coeftest(md, vcov = vcovHC(md, type="HC2"))
    }
    data.frame(
      outcome = outcome,
      treatment = exp_group,
      num = md[2, 1],
      se = md[2, 2],
      p_value = md[2, 4],
      N = length(survey_weights)
    )
  }


# Run the analysis
ar_est <- rbind(
  lapply(
    ar_groups[2:4],
    survey_experiment_func,
    outcome_var = "Q12_clean",
    outcome = "Agreement with statement that U.S. should invest more in AI military capabilities"
  ) %>% do.call(what = rbind),
  lapply(
    ar_groups[2:4],
    survey_experiment_func,
    outcome_var = "Q13_clean",
    outcome = "Agreement with statement that U.S. should work hard to cooperate with China to avoid dangers of AI arms race"
  ) %>% do.call(what = rbind)
)

# Improve the aesthetics
ar_est$treatment <-
  factor(ar_est$treatment, levels = rev(levels(ar_est$treatment)))
ar_est$outcome <- str_wrap(ar_est$outcome, width = 40)
ar_est$outcome <-
  factor(ar_est$outcome, levels = unique(ar_est$outcome))
ar_est$stars <- ""
ar_est$stars[ar_est$p_value < 0.05] <- "*"
ar_est$stars[ar_est$p_value < 0.005] <- "**"
ar_est$stars[ar_est$p_value < 0.001] <- "***"
ar_est$new_text <- paste0(roundfunc(ar_est$num), " (MOE = +/-",
                          roundfunc(ar_est$se*qnorm(0.975)), ")")

ggplot(data = ar_est,
       aes(
         x = treatment,
         y = num,
         ymin = qnorm(0.025) * se + num,
         ymax = qnorm(0.975) * se + num
       )) +
  geom_hline(yintercept = 0,
             linetype = 2,
             alpha = 0.5) +
  geom_pointrange(position = position_dodge(width = 0.9)) +
  geom_text(aes(label = new_text), nudge_x = 0.3, alpha = 0.6) +
  coord_flip() +
  scale_x_discrete(
    labels = function(x)
      str_wrap(x, width = 20),
    name = "Experimental groups"
  ) +
  scale_y_continuous(
    name = "Estimated treatment effects\n(Responses: -2 = Strongly disagree; 2 = Strongly agree)"
  ) +
  labs(caption = "Source: Center for the Governance of AI") +
  facet_grid( ~ outcome, labeller = label_wrap_gen(width = 45)) + theme_bw()
```

```{r armsracediff, echo=FALSE, fig.height=4, fig.keep='all', fig.cap="Difference in response to the two statements by experimental group", warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}
d$Q13_12_diff <- d$Q13_clean-d$Q12_clean
d$Q13_12_diff_missing <- !d$Q13_12_diff %in% c(-4:4)

# Difference between statements

ar_diff <- function(gnum) {
  md_o <- lm(Q13_12_diff ~ scale(Q13_12_diff_missing), data = d, 
           subset = d$q12_treat == gnum,
           weights = d$survey_weights)
  md <- coeftest(md_o, vcov = vcovHC(md_o, type="HC2"))
  return(data.frame(exp_group_l = gnum, num = md[1,1], 
                    se = md[1,2], N = length(md_o$residuals)))
}

ar_diff_res <- do.call(rbind, lapply(1:4, ar_diff))
ar_diff_res$exp_group_l <- ar_groups
ar_diff_res$sum_stat <- paste0(roundfunc(ar_diff_res$num), " (MOE: +/-",
              roundfunc(ar_diff_res$se*qnorm(0.975)), 
                               "); N = ", ar_diff_res$N)
ar_diff_res$exp_group_l <- factor(ar_diff_res$exp_group_l, levels = rev(ar_groups))

# Make the plot
ggplot(data = ar_diff_res, aes(x = exp_group_l, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_hline(yintercept = 0, alpha = 0.5, linetype = 2) +
  geom_pointrange(position = position_dodge(width = 0.9)) + 
  geom_text(aes(label = sum_stat), nudge_x = 0.4, alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 20),
                         name = "Experimental groups") + 
  expand_limits(x = c(1, 5)) +
  scale_y_continuous(
    name = "Response to Statement 2 (cooperate with China) -\nResponse to Statement 1 (invest in AI military capabilities)") + 
  labs(
       caption = "Source: Center for the Governance of AI") + 
  theme_bw()

```

## Americans see the potential for U.S.-China cooperation on some AI governance challenges

We examined issue areas where Americans perceive likely U.S.-China cooperation. Each respondent was randomly assigned to consider three out of five AI governance challenges. For each challenge, the respondent was asked, "For the following issues, how likely is it that the U.S. and China can cooperate?". (See [Appendix B](#uschinacoop) for the question text.)

On each of these AI governance issues, Americans see some potential for U.S.-China cooperation, according to Figure \@ref(fig:coopchina). U.S.-China cooperation on value alignment is perceived to be the most likely (48% mean likelihood). Cooperation to prevent AI-assisted surveillance that violates privacy and civil liberties is seen to be the least likely (40% mean likelihood) -- an unsurprising result since the U.S. and China take different stances on human rights. 

Despite current tensions between Washington and Beijing, the Chinese government, as well as Chinese companies and academics, have signaled their willingness to cooperate on some governance issues. These include banning the use of lethal autonomous weapons [@kania2018], building safe AI that is aligned with human values [@chinaai2018], and collaborating on research [@borderlessresearch]. Most recently, the major tech company Baidu became the first Chinese member of the Partnership on AI, a U.S.-based multi-stakeholder organization committed to understanding and discussing AI's impacts [@cadell2018]. 

In the future, we plan to survey Chinese respondents to understand how they view U.S.-China cooperation on AI and what governance issues they think the two countries could collaborate on. 

```{r coopchina, echo=FALSE, fig.height=4.5, width = 7, fig.keep='all', warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Issue areas for possible U.S.-China cooperation"}
# Overall mean to recode the missing data
coop_overall_mean <- wtd.mean(relabel_var(as.numeric(unlist(d[,paste0("Q14_", 1:5)])), 
            c(1:8, 98, 99), c(mc_p_med/100, NA, NA, NA)), 
         weights = rep(d$survey_weights, 5), na.rm = TRUE)

# Function to generate the results
china_coop <- function(item_num, output_type = "num_value",
                      new_values = c(mc_p_med/100, NA, NA, NA)) {
  # Generate the "shown" variable
  shown_variable <- d[,paste0("Q14_", item_num)] != 99
  # Analysis function
  return(catvar_func(
    outcome = label(x = d0[,paste0("Q14_", item_num)]),
    outcome_var = d[,paste0("Q14_", item_num)],
    label_var = d0[,paste0("Q14_", item_num)],
    output_type = output_type,
    shown = shown_variable,
    num_missing = 98,
    num_DK = 8, missing_recode = coop_overall_mean,
    new_values = new_values))
}
# Generate the data
china_cd_num_value <- do.call(rbind, lapply(1:5, china_coop))
# Clean the data
coop_outcome <- c("Prevent AI cyber attacks against governments, companies, organizations, and individuals", 
                                "Prevent AI-assisted surveillance from violating privacy and civil liberties", 
                                "Make sure AI systems are safe, trustworthy, and aligned with human values", "Ban the use of lethal autonomous weapons", 
                                "Guarantee a good standard of living for those who lose their jobs to automation")
china_cd_num_value$outcome <- coop_outcome
# Make the plot
china_cd_num_value$sum_stat <- paste0(roundfunc(china_cd_num_value$num*100, 1), " (MOE = +/-",
                                      roundfunc(100*china_cd_num_value$se*qnorm(0.975), 1), 
                                      "); N = ",
                                      china_cd_num_value$N)
ggplot(data = china_cd_num_value, aes(x = outcome, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_pointrange(position = position_dodge(width = 0.9)) + 
  geom_text(aes(label = sum_stat), 
            nudge_x = 0.3, alpha = 0.6) +
  coord_flip() + expand_limits(x = c(1, 6)) +
  scale_x_discrete(labels = function(x) str_wrap(x, width = 35),
                         name = "Issue areas") + 
  scale_y_continuous(limits = c(0.35, 0.525), labels = scales::percent,
    name = "Perceived likelihood of cooperation (percentage points)") + 
  labs(
       caption = "Source: Center for the Governance of AI") +
  theme_bw()
```

\newpage

# Trend across time: attitudes toward workplace automation

Survey questions measuring Americans' perceptions of workplace automation have existed since the 1950s. Our research seeks to track changes in these attitudes across time by connecting past survey data with original, contemporary survey data. 

## Americans do not think that labor market disruptions will increase with time

American government agencies, think tanks, and media organizations began conducting surveys to study public opinion about technological unemployment during the 1980s when unemployment was relatively high. Between 1983 and 2003, the U.S. National Science Foundation (NSF) conducted eight surveys that asked respondents the following:

>In general, computers and factory automation will create more jobs than they will eliminate.  Do you strongly agree, agree, disagree, or strongly disagree?

Our survey continued this time trend study by posing a similar -- but updated -- question (see [Appendix B](#jobtime)):

>Do you strongly agree, agree, disagree, or strongly disagree with the statement below?

>In general, automation and AI will create more jobs than they will eliminate.

Our survey question also addressed the chief ambiguity of the original question: lack of a future time frame. We used a survey experiment to help resolve this ambiguity by randomly assigning respondents to one of four conditions. We created three treatment conditions with the future time frames of 10 years, 20 years, and 50 years, as well as a control condition that did not specify a future time frame.

On average, Americans disagree with the statement more than they agree with it, although about a quarter of respondents in each experimental group give "don't know" responses. Respondents' agreement with the statement seems to increase slightly with the future time frame, but formal tests in [Apppendix C](#appjobloss) reveal that there exist no significant differences between the responses to the differing future time frames. This result is puzzling from the perspective that AI and robotics will increasingly automate tasks currently done by humans. Such a view would expect more _disagreement_ with the statement as one looks further into the future. One hypothesis to explain our results is that respondents believe the disruption from automation is destabilizing in the upcoming 10 years but eventually institutions will adapt and the labor market will stabilize. This hypothesis is consistent with our other finding that the median American predicts a 54% chance of high-level machine intelligence being developed within the next 10 years.

```{r jobsloss, echo=FALSE, fig.height=4.5, fig.keep='all', warning=FALSE, fig.cap="Agreement with the statement that automation and AI will create more jobs than it will eliminate", cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}

# Overall mean for recoding the missing values
job_creation_overall_mean <- 
  wtd.mean(relabel_var(d$Q15, c(1:5, 8, 9), c(2, 1, -1, -2, NA, NA, NA)),
           weights = d$survey_weights, na.rm = TRUE)

# Function to generate the results
job_creation <- function(varname, exp_group, output_type = "num_value",
                      new_values = c(2, 1, -1, -2, NA, NA, NA)) {
  return(data.frame(catvar_func(
  outcome = "Agreement with the statement that automation will lead to job creation",
  outcome_var = d[,varname],
  label_var = d0[,varname],
  output_type = output_type,
  shown = (d$q15_treat == exp_group),
  num_missing = 5,
  num_DK = 8,
  new_values = new_values, 
  missing_recode = job_creation_overall_mean,
  ), exp_group = names(val_labels(d0$q15_treat))[exp_group]))  
}
job_exp <- lapply(1:4, job_creation, varname = "Q15") %>% do.call(what = rbind)
job_exp$exp_group <- factor(job_exp$exp_group, levels = rev(levels(job_exp$exp_group)))
# Make the plot
job_exp$sum_stat_s <- gsub(pattern = "Mean: ", replacement = "",
                         job_exp$sum_stat)
ggplot(data = job_exp, aes(x = exp_group, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_pointrange(position = position_dodge(width = 0.9)) + 
  geom_text(aes(label = sum_stat_s), nudge_x = 0.3, alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 35),
                         name = "Time frame into the future") + 
  scale_y_continuous(
    name = "Agreement with the statement\n(-2 = Strongly disagree; 2 = Strongly agree)") + 
  labs(
       caption = "Source: Center for the Governance of AI") +
  theme_bw()

# Clean the data for regression analysis
d$Q15_clean <- relabel_var(d$Q15, c(1:5, 8, 9), c(2, 1, -1, -2, NA, NA, NA))
d$Q15_missing <- is.na(d$Q15_clean)
d$Q15_clean[is.na(d$Q15_clean)] <- job_creation_overall_mean
d$q15_treat_clean <- relabel_var(d$q15_treat, c(1:4, 8, 9), 
                                 c("No time frame", "10 years", 
                                   "20 years", "50 years", NA, NA))
time_exp <- c("No time frame", 
                                                  "10 years", 
                                   "20 years", "50 years")
d$q15_treat_clean <- factor(d$q15_treat_clean, levels = time_exp)

```

## Extending the historical time trend

The percentage of Americans that disagrees with the statement that automation and AI will create more jobs than they destroy is similar to the historical rate of disagreement with the same statement about computers and factory automation. Nevertheless, the percentage who agree with the statement has decreased by 12 percentage points since 2003 while the percentage who responded "don't know" has increased by 18 percentage points since 2003, according to Figure \@ref(fig:jobscompare).

There are three possible reasons for these observed changes. First, we have updated the question to ask about "automation and AI" instead of "computers and factory automation." The technologies we asked about could impact a wider swath of the economy; therefore, respondents may be more uncertain about AI's impact on the labor market. Second, there is a difference in survey mode between the historical data and our data. The NSF surveys were conducted via telephone while our survey is conducted online. Some previous research has shown that online surveys, compared with telephone surveys, produce a greater percentage of "don't know" responses [@nagelhout2010web; @bronner2007live]. But, other studies have shown that online surveys cause no such effect [@shin2012survey; @bech2009differential]. Third, the changes in the responses could be due to the actual changes in respondents' perceptions of workplace automation over time. 

```{r jobscompare, echo=FALSE, fig.height=5, fig.keep='all', warning=FALSE, cache=TRUE, dpi = 300, fig.cap="Response to statement that automation will create more jobs than it will eliminate[^jobsqfn] (data from before 2018 from National Science Foundation surveys)", dev = 'pdf', fig.width=7}

roper <- read.csv(paste0(wd, "roper_survey_data.csv"), stringsAsFactors=FALSE)
nsf <- roper[roper$QuestionTxt == "In general, computers and factory automation will create more jobs than they will eliminate.  Do you strongly agree, agree, disagree, or strongly disagree?", ]
# get the survey year
nsf$survey_year <- rep(NA, nrow(nsf))
for (i in 1:nrow(nsf)) {
  nsf$survey_year[i] <- as.numeric(strsplit(nsf$EndDate[i], "/")[[1]][3])
}
nsf$RespPct[nsf$RespPct == "*"] <- 0.5
nsf$RespPct <- as.numeric(nsf$RespPct)
# Add up agrees and disagrees
nsf <- nsf %>% group_by(QuestionID) %>% dplyr::summarize(
  Agree = sum(RespPct[RespTxt %in% c("Strongly agree", "Agree")]),
  Disagree = sum(RespPct[RespTxt %in% c("Strongly disagree", "Disagree")]),
  DK = sum(RespPct[RespTxt == "Don't know"]),
  survey_year = mean(survey_year)
) 
# Melt the data
nsf <- reshape2::melt(nsf, id = c("QuestionID", "survey_year"))
nsf$variable <- as.character(nsf$variable)
nsf$variable[nsf$variable == "DK"] <- "Don't know"
nsf$variable[nsf$variable == "Agree"] <- "Strongly agree/Agree"
nsf$variable[nsf$variable == "Disagree"] <- "Strongly disagree/Disagree"
nsf$variable <- factor(nsf$variable, 
                                  levels = unique(nsf$variable))
nsf$survey_type <- "NSF"
# Add in the YouGov data
job_exp_nsf <- lapply(1:4, job_creation, varname = "Q15", 
                      output_type = "value_table") %>% do.call(what = rbind) %>% group_by(exp_group) %>% dplyr::summarise(
    Agree = sum(Prop[labels %in% c("2. Strongly agree", "1. Agree")])*100,
    Disagree = sum(Prop[labels %in% c("-2. Strongly disagree", "-1. Disagree")])*100,
    DK = sum(Prop[labels == "Don't know"])*100,
    survey_year = 2018,
    survey_type = "Gov-AI"
)
names(job_exp_nsf)[names(job_exp_nsf) == "exp_group"] <- "QuestionID"
job_exp_nsf <- reshape2::melt(job_exp_nsf, id = c("QuestionID", 
                                                  "survey_year", "survey_type"))
job_exp_nsf$variable <- as.character(job_exp_nsf$variable)
job_exp_nsf$variable[job_exp_nsf$variable == "DK"] <- "Don't know"
job_exp_nsf$variable[job_exp_nsf$variable == "Agree"] <- "Strongly agree/Agree"
job_exp_nsf$variable[job_exp_nsf$variable == "Disagree"] <- "Strongly disagree/Disagree"
job_exp_nsf$variable <- factor(job_exp_nsf$variable, 
                                  levels = unique(job_exp_nsf$variable))
nsf <- rbind(nsf, job_exp_nsf)
nsf$noexp <- !nsf$QuestionID %in% c("10 years", "20 years", "50 years")
# Make the graph
ggplot() + 
  geom_point(data = nsf[nsf$noexp,], 
             aes(x = survey_year, y = value/100, color = variable)) + 
  geom_point(data = nsf[!nsf$noexp,], 
             aes(x = survey_year, y = value/100, shape = QuestionID, color = variable)) + 
  geom_line(data = nsf[nsf$noexp,], 
            aes(x = survey_year, y = value/100, color = variable)) +
  geom_text(data = nsf[nsf$noexp,], 
            aes(x = survey_year, y = value/100, label = roundfunc(value, 0)),
            size = 3, nudge_y = 0.02) +
  scale_x_continuous("Survey year",
                     breaks = seq(1980, 2020, by = 5)) + 
  scale_color_discrete(name = "Responses") +
  scale_shape_manual(name = "Time frames for experimental conditions", 
                     values = c(22, 23, 24)) + 
  scale_y_continuous(name = "Percentage of respondents", labels = scales::percent) + 
  theme_bw() + theme(legend.position = "bottom", legend.direction = "vertical") +
  labs(caption = "Source: Center for the Governance of AI; NSF")

```

[^jobsqfn]: Note that our survey asked respondents this question with the time frames 10, 20 and 50 years, whereas the NSF surveys provided no time frame.  

\newpage

# High-level machine intelligence


## The public predicts a 54% likelihood of high-level machine intelligence within 10 years {#arrivesooner}

Respondents were asked to forecast when high-level machine intelligence will be developed. High-level machine intelligence was defined as the following:

>We have high-level machine intelligence when machines are able to perform almost all tasks that are economically relevant today better than the median human (today) at each task. These tasks include asking subtle common-sense questions such as those that travel agents would ask. For the following questions, you should ignore tasks that are legally or culturally restricted to humans, such as serving on a jury. [^hlmidef]

[^hlmidef]: Note that our definition of high-level machine intelligence is equivalent to what many would consider human-level machine intelligence. Details of the question are found in [Appendix B](#forecasthlmi).

Respondents were asked to predict the probability that high-level machine intelligence will be built in 10, 20, and 50 years.

We present our survey results in two ways. First, we show the summary statistics in a simple table. Next, to compare the public's forecasts with forecasts made by AI researchers in 2016 [@grace2018will], we aggregated the respondents' forecasts using the same method. Note that @grace2018will gave a stricter definition of high-level machine intelligence that involved machines being better than all humans at all tasks. [^gracemethodfn] 

```{r hlmitimelinetablesum, echo=FALSE, fig.height=5.75, fig.keep='all',  warning=FALSE, cache=TRUE, results=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="The American public's forecasts of high-level machine intelligence timelines"}

# Clean the data
# Get the overall mean for each year
hlmi_clean <- function(dataset = d) {
  # Within 10 years
  hlmi_10 <-
    relabel_var(
      dataset$Q16_1,
      old_labels = c(1:8, 98, 99),
      new_labels = c(mc_p_med / 100, NA, NA, NA)
    )
  hlmi_10_missing <- is.na(hlmi_10)
  hlmi_10_overall_mean <- wtd.mean(hlmi_10[!is.na(hlmi_10)],
                                   weights = dataset$survey_weights[!is.na(hlmi_10)])
  hlmi_10[is.na(hlmi_10)] <- hlmi_10_overall_mean
  # Within 20 years
  hlmi_20 <- relabel_var(
    dataset$Q16_2,
    old_labels = c(1:8, 98, 99),
    new_labels = c(mc_p_med / 100, NA, NA, NA)
  )
  hlmi_20_missing <- is.na(hlmi_20)
  hlmi_20_overall_mean <- wtd.mean(hlmi_20[!is.na(hlmi_20)],
                                   weights = dataset$survey_weights[!is.na(hlmi_20)])
  hlmi_20[is.na(hlmi_20)] <- hlmi_20_overall_mean
  # Within 50 years
  hlmi_50 <- relabel_var(
    dataset$Q16_3,
    old_labels = c(1:8, 98, 99),
    new_labels = c(mc_p_med / 100, NA, NA, NA)
  )
  hlmi_50_missing <- is.na(hlmi_50)
  hlmi_50_overall_mean <- wtd.mean(hlmi_50[!is.na(hlmi_50)],
                                   weights = dataset$survey_weights[!is.na(hlmi_50)])
  hlmi_50[is.na(hlmi_50)] <- hlmi_50_overall_mean
  return(
    data.frame(
      hlmi_10,
      hlmi_20,
      hlmi_50,
      hlmi_10_missing,
      hlmi_20_missing,
      hlmi_50_missing
    )
  )
}

# HLMI full data
hlmi_full <- hlmi_clean(dataset = d)
d <- cbind(d, hlmi_full)
# check monotone increase
full_mono_check <- mean(hlmi_full$hlmi_10 <= hlmi_full$hlmi_20 & 
          hlmi_full$hlmi_10 <= hlmi_full$hlmi_50 &
          hlmi_full$hlmi_20 <= hlmi_full$hlmi_50)
# HLMI people with CS/engineering degrees
hlmi_cs <- hlmi_full[d$demo_cs == "CS or engineering degree",]
# check monotone increase
cs_mono_check <- mean(hlmi_cs$hlmi_10 <= hlmi_cs$hlmi_20 & 
          hlmi_cs$hlmi_10 <= hlmi_cs$hlmi_50 &
            hlmi_cs$hlmi_20 <= hlmi_cs$hlmi_50)

# Helper function
hlmi_timeline <- function(year_num, missing_recode, output_type = "num_value",
                          shown = rep(TRUE, nrow(d))) {
  catvar_func(
    outcome = label(x = d0[,paste0("Q16_", year_num)]),
    outcome_var = d[,paste0("Q16_", year_num)],
    label_var = d0[,paste0("Q16_", year_num)],
    output_type = output_type,
    shown = shown,
    num_missing = 8,
    num_DK = 98, missing_recode = missing_recode, 
    new_values <- c(mc_p_med/100, NA, NA, NA))  
}

# Make summary statistics table
hlmi_sum <- function(varname, subset = rep(TRUE, nrow(d))) {
  x = d[,varname]
  q_res <- wtd.quantile(x[subset], weights = d$survey_weights[subset], probs=c(.25, .5, .75))
  m_res <- wtd.mean(x[subset], weights = d$survey_weights[subset])
  N <- sum(as.numeric(subset))
  res_df <- round(c(as.numeric(c(q_res[1:2], m_res, q_res[3]))*100, N))
  names(res_df) <- c("Q1", "Median", "Mean", "Q3", "N")
  return(res_df)
}

# Make the overall table
hlmi_year = c(10, 20, 50) + 2018

demo_hlmi <- function(demo_var, demo_group, demo_value) {
  data.frame(hlmi_year, demo_group, demo_value, 
             do.call(rbind, lapply(paste0("hlmi_", c(10, 20, 50)), hlmi_sum, 
                                   subset = d[,demo_var] == demo_value)))  
}

demo_hlmi_loop <- function(demo_var, demo_group) {
  levels_demo <- levels(d[,demo_var])
  lapply(levels_demo, demo_hlmi, demo_var = demo_var,
         demo_group = demo_group) %>% do.call(what = rbind)
}

# Make the summary statistics
demo_hlmi_loop_res <- rbind(
  demo_hlmi_loop(demo_var = "demo_age", demo_group = "Age group"),
  demo_hlmi_loop(demo_var = "demo_gender", demo_group = "Gender"),
  demo_hlmi_loop(demo_var = "demo_white", demo_group = "Race"),
  demo_hlmi_loop(demo_var = "demo_educ", demo_group = "Education"),
  demo_hlmi_loop(demo_var = "demo_employ", demo_group = "Employment status"),
  demo_hlmi_loop(demo_var = "demo_income", demo_group = "Income"),
  demo_hlmi_loop(demo_var = "demo_pid3", demo_group = "Political party"),
  demo_hlmi_loop(demo_var = "demo_rel", demo_group = "Religion"),
  demo_hlmi_loop(demo_var = "demo_bornagain", demo_group = "Born-again Christian"),
  demo_hlmi_loop(demo_var = "demo_cs", demo_group = "CS or engineering degree"),
  demo_hlmi_loop(demo_var = "demo_prog", demo_group = "CS or programming experience"))

hlmi_all <- data.frame(hlmi_year, demo_value = "All respondents", do.call(rbind, 
                              lapply(paste0("hlmi_", c(10, 20, 50)), hlmi_sum)))

hlmi_cs <- demo_hlmi_loop_res[grep(pattern = "CS or engineering degree", 
                        x = demo_hlmi_loop_res$demo_group, ignore.case = FALSE),
                   c("hlmi_year", "demo_value", "Q1", "Median", "Mean", "Q3", "N")]

hlmi_table <- rbind(hlmi_all, hlmi_cs)
hlmi_table$Q1 <- paste0(hlmi_table$Q1, "%")
hlmi_table$Median <- paste0(hlmi_table$Median, "%")
hlmi_table$Mean <- paste0(hlmi_table$Mean, "%")
hlmi_table$Q3 <- paste0(hlmi_table$Q3, "%")

kable(hlmi_table,
    caption = "Summary statistics of high-level machine intelligence forecast",
    format = "pandoc",
    col.names = c(
      "Year", "Respondent type",
      "25th percentile",
      "Median",
      "Mean",
      "75th percentile", "\\textit{N}"
    ), row.names = FALSE
  )

```

```{r hlmitimelinetabledemo, echo=FALSE, fig.height=5.75, fig.keep='all',  warning=FALSE, cache=TRUE, results=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="The American public's forecasts of high-level machine intelligence timelines"}

# demo_hlmi_loop_res$demo_value <- factor(demo_hlmi_loop_res$demo_value, 
#                                         levels = rev(unique(demo_hlmi_loop_res$demo_value)))
# ggplot(demo_hlmi_loop_res, aes(x = demo_value, y = Mean/100, shape = factor(hlmi_year),
#                                 color = factor(hlmi_year))) +
#   geom_point() + coord_flip() + theme_bw() +
#   scale_y_continuous(labels = scales::percent, name = "Mean likelihood") + 
#   scale_shape_discrete(name = "Year") + scale_color_discrete(name = "Year") +
#   theme(legend.position = "bottom")
```

[^gracemethodfn]: In @grace2018will, each respondent provides three data points for their forecast, and these are fitted to the Gamma CDF by least squares to produce the individual cumulative distribution function (CDFs). Each "aggregate forecast" is the mean distribution over all individual CDFs (also called the "mixture" distribution). The confidence interval is generated by bootstrapping (clustering on respondents) and plotting the 95% interval for estimated probabilities at each year. Survey weights are not used in this analysis due to problems incorporating survey weights into the bootstrap.  

Respondents predict that high-level machine intelligence will arrive fairly quickly. The median respondent predicts a likelihood of 54% by 2028, a likelihood of 70% by 2038, and a likelihood of 88% by 2068, according to Table \@ref(tab:hlmitimelinetablesum).

These predictions are considerably sooner than the predictions by experts in two previous surveys. In @muller2014future, expert respondents predict a 50% probability of high-level human intelligence being developed by 2040-2050 and 90% by 2075. In @grace2018will, experts predict that there is a 50% chance that high-level machine intelligence will be built by 2061. Plotting the public's forecast with the expert forecast from @grace2018will, we see that the public predicts high-level machine intelligence arriving much sooner than experts forecast. Employing the same aggregation method used in @grace2018will, Americans predict that there is a 50% chance that high-level machine intelligence will be developed by 2026. 

Results in @walsh2018expert also show that the non-experts (i.e., readers of a news article about AI) are more optimistic in their predictions of high-level machine intelligence compared with experts. In Walsh's study, the median AI expert predicted a 50% probability of high-level machine intelligence by 2061 while the median non-expert predicted a 50% probability by 2039. In our survey, respondents with CS or engineering degrees, compared with those who do not, provide a somewhat longer timeline for the arrival of high-level machine intelligence, according to Table \@ref(tab:hlmitimelinetablesum). Nevertheless, those with CS or engineering degrees in our sample provide forecasts are more optimistic than those made by experts from @grace2018will; furthermore, their forecasts show considerable overlap with the overall public forecast (see Figure \@ref(fig:hlmifigure)).

The above differences could be due to different definitions of high-level machine intelligence presented to respondents. However, we suspect that it is not the case for the following reasons. (1) These differences in timelines are larger, more significant than we think could be reasonably attributed to beliefs about these different levels of intelligence. (2) We found similar results using the definition in @grace2018will, on a (different) sample of the American public. In a pilot survey conducted on Mechanical Turk during July 13-14, 2017, we asked American respondents about human-level AI, defined as the following:

>Human-level artificial intelligence (human-level AI) refers to computer systems that can operate with the intelligence of an average human being. These programs can complete tasks or make decisions as successfully as the average human can.

In this pilot study, respondents also provided forecasts that are more optimistic than the projections by AI experts. The respondents predict a median probability of 44% by 2027, a median probability of 62% by 2037, and a median probability of 83% by 2067.

```{r hlmifigure, echo=FALSE, fig.height=5.75, fig.keep='all',  warning=FALSE, cache=TRUE, results=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="The American public's forecasts of high-level machine intelligence timelines"}
# Make the CDF graph
# Combine the data for making the CDF graph
hlmi_d <- rbind(data.frame(response.id = d$r_id, 
                           fixedprobabilities = 0, field.group = "Public",
           p = hlmi_full$hlmi_10, x = 10, survey_weights = d$survey_weights),
      data.frame(response.id = d$r_id, fixedprobabilities = 0, field.group = "Public",
           p = hlmi_full$hlmi_20, x = 20, survey_weights = d$survey_weights),
      data.frame(response.id = d$r_id, fixedprobabilities = 0, field.group = "Public",
           p = hlmi_full$hlmi_50, x = 50, survey_weights = d$survey_weights))

# Initial parameter values
gamma.inits <- list(
    c(4,1),
    c(4,4),
    c(4,16),
    c(4,64),
    c(4,256),
    c(4,1096),
    c(4, 4000),
    c(4, 16000))
# Fit to CDFs
gamma.fits <- hlmi_d %>% fit.all(gamma.dist.f, par_init = gamma.inits)
gamma.fits$weights <- d$survey_weights
x <- seq(0, 1000, by = 0.2)
gamma.fits.cs <- gamma.fits[d$demo_cs == "CS or engineering degree",]

# All respondents
curves <- gamma.fits %>% cum.dist(x, gamma.dist.f) %>% rename(V1=response.id, y=p)
curves$V1 <- paste0("Respondent_", curves$V1)
curves$shape <- NULL
curves$scale <- NULL
combined.curve <- curves %>% 
    group_by(x) %>% 
    dplyr::summarize(y=mean(y), V1="summary_cdf")
# Combine the data for plotting
pdata <- bind_rows(
            forecasters = curves,
            combined = combined.curve, 
            .id = "source")

make.curve.matrix <- function(curves) {
    curves %>% 
        dplyr::select(V1, x, y) %>% 
        spread(x, y) %>% 
        dplyr::select(-V1) %>%
        data.matrix
}

curve.matrix <- make.curve.matrix(curves)

bootstrap.curve.matrix <- function(curve.matrix, n.boot) { 
    n.r = curve.matrix %>% nrow
    bootstrap.samples = rmultinom(n = n.boot, size = n.r, 
                                  prob = rep(1, n.r)) %>% t
    bootstrap.curves = (bootstrap.samples / n.r) %*% curve.matrix
    
    flat.bootstrap.curves = bootstrap.curves %>% 
        as_data_frame %>% 
        mutate(sample=1:nrow(.)) %>% 
        gather(x, y, -sample) %>% 
        mutate(x=as.numeric(x))
    
    flat.bootstrap.curves %>% 
    group_by(x) %>% 
    dplyr::summarize(lower = quantile(y, .025),upper = quantile(y, .975))
}

bs_res <- bootstrap.curve.matrix(curve.matrix, 2000)
# plot 50 forecaster's predictions (not too many)
plot_keep <- unique(pdata$V1)[sample(x = 1:length(unique(pdata$V1)), 
                                     size = 50)]

# Respondents with CS/engineering degrees
d$V1 <- paste0("Respondent_", d$r_id)
curves.cs <- subset(curves, V1 %in% d$V1[d$demo_cs == "CS or engineering degree"])
combined.curve.cs <- curves.cs %>% 
    group_by(x) %>% 
    dplyr::summarize(y = mean(y), V1="summary_cdf_cs")
combined.curve.cs$source <- "cs_combined"
# Combine the data for plotting
pdata.cs <- bind_rows(
            forecasters = curves.cs,
            combined = combined.curve.cs, 
            .id = "source")
curve.matrix.cs <- make.curve.matrix(curves.cs)
bs_res.cs <- bootstrap.curve.matrix(curve.matrix = curve.matrix.cs, n.boot = 2000)


# Load in the expert forecasts
expert_bs <- read.csv(paste0(wd, "experts_forecasts.csv"))
# Change the forecast years starting from 2018
expert_bs$x <- expert_bs$x+2016-2018 
expert_lines <- read.csv(paste0(wd, "experts_forecasts_lines.csv"))
expert_lines$x <- expert_lines$x+2016-2018

# Merge in the data
pdata$source <- paste0("public_", pdata$source)
pdata$V1[pdata$V1 == "summary_cdf"] <- "public_summary_cdf"
expert_lines$source <- paste0("expert_", as.character(expert_lines$source))
expert_lines$V1 <- as.character(expert_lines$V1)
expert_lines$V1[expert_lines$V1 == "summary_cdf"] <- "expert_summary_cdf"
# Combine the public and expert forecasts
all_hlmi <- rbind(pdata[,c("source", "x", "V1", "y")],
                  combined.curve.cs[,c("source", "x", "V1", "y")],
                  expert_lines[expert_lines$source == "expert_combined",
                               c("source", "x", "V1", "y")])
# Make the plot: all data
#all_hlmi$source[all_hlmi$source == "public_public_public_combined"] <- "public_combined"
#all_hlmi$source[all_hlmi$source == "public_public_public_forecasters"] <- "public_forecasters"

line.labels = c(
  public_forecasters = "Random subset of survey respondents",
  public_combined = "Public aggregate forecast (with 95% confidence interval)",
  cs_combined = "Respondents with CS/engineering degrees aggregate forecast (with 95% confidence interval)",
  expert_combined = "Expert aggregate forecast (with 95% confidence interval)"
 )

line.color = c(
    public_forecasters = "grey",
    public_combined = "red",
    cs_combined = "darkblue",
    expert_combined = "blue")

line.alpha = c(
    public_forecasters = .5, 
    public_combined = 1,
    cs_combined = 1,
    expert_combined = 1)

line.size = c(
    public_forecasters = .2, 
    public_combined = 0.5,
    cs_combined = 0.5,
    expert_combined = 0.5)

line.linetype = c(
    public_forecasters = 1, 
    public_combined = 1,
    cs_combined = 1,
    expert_combined = 1)

pdata.df = all_hlmi %>% 
    filter(V1 %in% plot_keep | source %in% c("expert_combined", "public_combined",
                                             "cs_combined")) %>% 
    as.data.frame()

pdata.df$source <- factor(pdata.df$source, c("public_combined", "cs_combined",
                                      "expert_combined", "public_forecasters"))

# Make the graph 
ggplot() + 
  geom_ribbon(data = bs_res, mapping = aes(x = x + 2018, ymin = lower, ymax = upper),
              alpha = 0.4, fill = "red") +
  # geom_ribbon(data = expert_bs, mapping = aes(x = x + 2018, ymin = lower, ymax = upper),
  #             alpha = 0.4, fill = "blue") +
  geom_ribbon(data = bs_res.cs, mapping = aes(x = x + 2018, ymin = lower, ymax = upper),
              alpha = 0.4, fill = "blue") +
  geom_line(data = pdata.df[pdata.df$source != "expert_combined",], 
            mapping = aes(x=x + 2018, y=y, group=V1, color=source, alpha = source, 
                  size=source, linetype=source)) +
  scale_color_manual(values=rev(line.color), name = NULL, labels=rev(line.labels)) +
  scale_alpha_manual(values=rev(line.alpha), name = NULL, labels=rev(line.labels)) +
  scale_size_manual(values=rev(line.size), name = NULL, labels=rev(line.labels)) +
  scale_linetype_manual(values=rev(line.linetype), name = NULL, labels=rev(line.labels)) + 
  xlab("Year") + 
  ylab("Probability of high-level\nmachine intelligence") + 
  coord_cartesian(xlim = c(2018, 2018 + 100), ylim=c(0,1)) +
  scale_x_continuous(expand=c(0,0)) + 
  scale_y_continuous(expand=c(0,0)) + 
  theme_bw() + 
  theme(legend.position = "bottom", 
        legend.direction="vertical") +
  labs(
       caption = "Source: Center for the Governance of AI")

# Remove large objects
rm("curve.matrix")
rm("pdata")
rm("gamma.fits")
rm("hlmi_d")
rm("hlmi_full")
rm("curves")
rm("curve.matrix.cs")
```

## Americans express mixed support for developing high-level machine intelligence {#subsecsupporthlmi}


```{r supporthlmi, echo=FALSE, fig.height=4.5, results='hide', fig.keep='all', warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Support for developing high-level machine intelligence"}

# Get the overall mean for recoding missing data
support_hlmi_overall_mean <- wtd.mean(relabel_var(d$Q17,
                                      c(1:6, 8, 9),
                                      c(2, 1, 0, -1, -2, NA, NA, NA)),
                                      weights = d$survey_weights, na.rm = TRUE)

# Prepare the data
support_hlmi_value_table <- catvar_func(
    outcome = "Support for developing high-level machine intelligence",
    outcome_var = d$Q17,
    label_var = d0$Q17,
    output_type = "value_table",
    shown = rep(TRUE, nrow(d)),
    num_missing = 8,
    num_DK = 6, missing_recode = support_hlmi_overall_mean,
    new_values = c(2, 1, 0, -1, -2, 98, 99, 100))
support_hlmi_value_table$num <- support_hlmi_value_table$new_values
support_hlmi_value_sum <- catvar_func(
    outcome = "Support for developing high-level machine intelligence",
    outcome_var = d$Q17,
    label_var = d0$Q17,
    output_type = "num_value",
    shown = rep(TRUE, nrow(d)),
    num_missing = 8,
    num_DK = 6, missing_recode = support_hlmi_overall_mean,
    new_values = c(2, 1, 0, -1, -2, NA, NA, NA))
# Fix the frequency texts
support_hlmi_value_table$Prop_text <- 
  round(support_hlmi_value_table$Prop*100, digits = 0)
support_hlmi_value_table$Prop_text[support_hlmi_value_table$Prop_text == "0"] <- "<1"

# Make the graph
support_hlmi_value_table$labels <- factor(support_hlmi_value_table$labels, 
                          levels = support_hlmi_value_table$labels[c(5:1, 6:7)])
ggplot() +
  geom_bar(data = support_hlmi_value_table[support_hlmi_value_table$Prop !=0,], 
           aes(x = num, y = Prop, fill = labels), stat = "identity",
           alpha = 0.6, color = "black") +
  geom_errorbar(data = 
                  support_hlmi_value_table[support_hlmi_value_table$Prop !=0,], 
                aes(x = num, ymin = Prop + qnorm(0.025)*se,
                    ymax = Prop + qnorm(0.975)*se), width = 0.1) +
  geom_text(data = support_hlmi_value_table, aes(x = num, 
                                           label = Prop_text), 
            y = 0.02, nudge_x = 0.25) +
  scale_x_continuous(breaks = support_hlmi_value_table$num[order(support_hlmi_value_table$num)],
    labels = str_wrap(support_hlmi_value_table$labels[order(support_hlmi_value_table$num)], 
                      width = 15)) +
  facet_grid(~group, scales = "free_x", space = "free_x") + theme_bw() +
  geom_text(data = support_hlmi_value_sum, aes(x = 0, label = sum_stat,
                                       y = max(support_hlmi_value_table$Prop)+0.05)) +
  scale_y_continuous(labels = scales::percent, 
                     limits = c(0, max(support_hlmi_value_table$Prop)+0.05)) +
      scale_fill_manual(values = c("#ca0020", "#f4a582", "#f7f7f7", "#92c5de", "#0571b0",
                               "grey65", "grey65")) + 
  xlab("Responses") + ylab("Percentage of respondents") + 
  labs(
       source = "Governance of AI Program") + theme(legend.position = "none")

# Correlation between support for developing AI and support for developing HLMI
# AI 
d$Q5_clean <- relabel_var(old_var = d$Q5, old_labels = c(1:6, 8, 9),
                          new_labels = c(2, 1, 0, -1, -2, NA, NA, NA))
d$Q5_missing <- is.na(d$Q5_clean)
d$Q5_clean[is.na(d$Q5_clean)] <- dev_ai_overall_mean
# HLMI
d$Q17_clean <- relabel_var(d$Q17, c(1:6, 8, 9), c(2, 1, 0, -1, -2, NA, NA, NA))
d$Q17_missing <- is.na(d$Q17_clean)
d$Q17_clean[is.na(d$Q17_clean)] <- support_hlmi_overall_mean
cor(d$Q5_clean, d$Q17_clean)
# Linear regression
support_d <- reshape2::melt(d[,c("Q5_clean", "Q17_clean", "survey_weights",
                                 "Q17_missing", "r_id")], 
                            id = c("r_id", "Q17_missing", "survey_weights"))
support_d$variable <- ifelse(support_d$variable == "Q5_clean", "AI", 
                             "High-level machine intelligence")
support_diff <- lm(value ~ variable + scale(Q17_missing) +
                     variable:scale(Q17_missing), 
                   data = support_d, weights = support_d$survey_weights)

rm("support_d")
```


Respondents were asked how much they support or oppose the development of high-level machine intelligence. (See [Appendix B](#supporthlmi) for the question text.) Americans express mixed support for developing high-level machine intelligence, much like how they feel about developing AI. About one-third of Americans (31%) somewhat or strongly support the development of high-level machine intelligence, while 27% somewhat or strongly oppose it. [^comment31] Many express a neutral attitude: 29% state that they neither support nor oppose, while 12% indicate they don't know.

[^comment31]: The discrepancy between this figure and the percentages in Figure \@ref(fig:supporthlmi) is due to rounding. According to Table \@ref(tab:hlmisupportdevtable), 7.78% strongly support and 23.58% somewhat support; therefore, 31.36% -- rounding to 31% -- of respondents either support or somewhat support. 

The correlation between support for developing AI and support for developing high-level machine intelligence is 0.61. The mean level of support for developing high-level machine intelligence, compared with the mean level of support for developing AI, is 0.24 points (MOE = +/- 0.04) lower on a five-point scale (two-sided $p$-value $<0.001$), according to Table \@ref(tab:diffsupportaihlmi).

## High-income Americans, men, and those with tech experience express greater support for high-level machine intelligence {#subsecdemohlmi}

Support for developing high-level machine intelligence varies greatly between demographic subgroups, although only a minority in each subgroup supports developing the technology. Some of the demographic trends we observe regarding support for developing AI also are evident regarding support for high-level machine intelligence. Men (compared with women), high-income Americans (compared with low-income Americans), and those with tech experience (compared with those without) express greater support for high-level machine intelligence.

We used a multiple regression that includes all of the demographic variables to predict support for developing high-level machine intelligence. The support for developing AI outcome variable was standardized, so it has mean 0 and unit variance. 

Significant predictors correlated with support for developing high-level machine intelligence include:

- Being male (versus being female)
- Identifying as a Republican (versus identifying as an Independent or "other") [^repdemind]
- Having a family income of more than \$100,000 annually (versus having a family income of less than \$30,000 annually)
- Having CS or programming experience (versus not having such experience)

[^repdemind]: In the survey, we allowed those who did not identify as Republican, Democrat, or Independent to select "other." The difference in responses between Republicans and Democrats is not statistically significant at the 5% level. Nevertheless, we caution against over-interpreting these results related to respondents' political identification because the estimated differences are substantively small while the correlating confidence intervals are wide.   

This last result about women less supportive of developing high-level machine intelligence than men is noteworthy as it speaks to the contrary claim sometimes made that it is primarily men who are concerned about the risks from advanced AI. Men are argued to be disproportionately worried about human-level AI because of reasons related to evolutionary psychology [@pinker2018enlightenment] or because they have the privilege of not confronting the other harms from AI, such as biased algorithms [@crawford2016].

We also performed the analysis above but controlling for respondents' support for developing AI (see [Appendix ](#appsupporthlmi)). Doing so allows us to identify subgroups those attitudes toward AI diverges from their attitudes toward high-level machine intelligence. In this secondary analysis, we find that being 73 or older is a significant predictor of _support_ for developing high-level machine intelligence. In contrast, having a four-year college degree is a significant predictor of _opposition_ to developing high-level machine intelligence. These are interesting inversions of the bivariate association, where older and less educated respondents were more concerned about AI; future work could explore this nuance.  


```{r demosupporthlmi1, echo=FALSE, fig.height=10, fig.keep='all',  warning=FALSE, cache=TRUE, dpi = 300, fig.width=8, dev = 'pdf', fig.cap="Support for developing high-level machine intelligence across demographic characteristics: distribution of responses"}

demo_support_values <- function(demo, demo_group, output_type = "value_sum") {
  levels_demo <- levels(d[,demo])
  lapply(levels_demo, demo_support, demo = demo, outcome_var = d$Q17, label_var = d0$Q17,
         demo_group = demo_group, output_type = output_type) %>% do.call(what = rbind)
}

# Make the value frequency table
d_value_table_support <- rbind(
  demo_support_values(demo = "demo_age", demo_group = "Age group",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_gender", demo_group = "Gender",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_white", demo_group = "Race",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_educ", demo_group = "Education",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_employ", demo_group = "Employment status",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_income", demo_group = "Income",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_pid3", demo_group = "Political party",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_rel", demo_group = "Religion",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_bornagain", demo_group = "Born-again Christian",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_cs", demo_group = "CS or engineering degree",
                      output_type = "value_table"),
  demo_support_values(demo = "demo_prog", demo_group = "CS or programming experience",
                      output_type = "value_table"))

# Make stacked bar chart
d_value_table_support$labels[d_value_table_support$labels %in%
                               c("I don't know", "Skipped")] <- 
  "Don't know/Skipped"
# Clean up the data
d_value_table_support <- 
  d_value_table_support %>% group_by(demo_value, demo_group, labels) %>%
  dplyr::summarise(Prop = sum(Prop))

# Make the graph
# Change the factor levels
d_value_table_support$demo_value <- factor(d_value_table_support$demo_value,
                                           levels = rev(levels(d_value_table_support$demo_value)))
# Change the factors
d_value_table_support$labels <- factor(d_value_table_support$labels,
     levels = rev(c("2. Strongly support", 
                "1. Somewhat support",
                "0. Neither support nor oppose", "-1. Somewhat oppose",
                "-2. Strongly oppose", "Don't know/Skipped")))
d_value_table_support$percent <- 
  roundfunc(d_value_table_support$Prop*100, 0)
# Make the graph
ggplot(data = d_value_table_support,
       aes(x=demo_value, y=Prop, fill=labels)) +
  geom_bar(stat="identity", position = "fill", alpha = 0.6) +
  geom_text(aes(label = percent),
            position = position_stack(vjust = 0.5), size = 3) +
  xlab("Demographic subgroups") +
  scale_y_continuous(name = "Percentage of respondents", labels = scales::percent,
                     limits = c(0, 1), expand = c(0, 0)) + coord_flip() +
  theme_bw() + theme(legend.position = "bottom") +
  facet_grid(demo_group~., scales = "free_y", space = "free_y") +
  scale_fill_manual(values = c("grey65", "#ca0020", "#f4a582", "#f7f7f7", "#92c5de", "#0571b0"), name = "Responses") +
   guides(fill = guide_legend(reverse = TRUE, nrow =  3)) +
  labs(
       caption = "Source: Center for the Governance of AI") +
  theme(strip.background = element_blank(),
   strip.text.y = element_blank(),
   axis.text.x = element_text(hjust=1))

```

```{r demosupporthlmi1b, echo=FALSE, fig.height=9.5, fig.keep='all',  warning=FALSE, cache=TRUE, dpi = 300, fig.width=7, dev = 'pdf', fig.cap="Support for developing high-level machine intelligence across demographic characteristics: average support across groups"}

# Make the summary statistics
d_value_sum_support <- rbind(
  demo_support_values(demo = "demo_age", demo_group = "Age group"),
  demo_support_values(demo = "demo_gender", demo_group = "Gender"),
  demo_support_values(demo = "demo_white", demo_group = "Race"),
  demo_support_values(demo = "demo_educ", demo_group = "Education"),
  demo_support_values(demo = "demo_employ", demo_group = "Employment status"),
  demo_support_values(demo = "demo_income", demo_group = "Income"),
  demo_support_values(demo = "demo_pid3", demo_group = "Political party"),
  demo_support_values(demo = "demo_rel", demo_group = "Religion"),
  demo_support_values(demo = "demo_bornagain", demo_group = "Born-again Christian"),
  demo_support_values(demo = "demo_cs", demo_group = "CS or engineering degree"),
  demo_support_values(demo = "demo_prog", demo_group = "CS or programming experience"))

# Clean the dataset
d_value_sum_support$demo_value <- factor(d_value_sum_support$demo_value,
                                         levels = rev(d_value_sum_support$demo_value))
shorten_sum <- function(x) {
  strsplit(as.character(x), split = ";")[[1]][1]  
}
d_value_sum_support$sum_stat_short <-
  do.call(rbind, lapply(d_value_sum_support$sum_stat, shorten_sum))
  
# Generate graph
ggplot(data = d_value_sum_support, aes(x = demo_value, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  #geom_hline(yintercept = 0, linetype = 2, alpha = 0.5) +
  geom_pointrange(position = position_dodge(width = 0.9), size = 0.25) + 
  geom_text(aes(y = num, label = roundfunc(num)), nudge_x = 0.3, nudge_y = 0.03, size = 2.75,
            alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
    name = "Demographic characteristics (grouped by demographic variable)") + 
  scale_y_continuous(
    name = "Support for developing AI (-2 = Strongly oppose; 2 = Strongly support)") + 
   facet_grid(demo_group~., scales = "free_y", space = "free_y") +
  labs(
       caption = "Source: Center for the Governance of AI") + 
  theme_bw() +
  theme(strip.background = element_blank(),
   strip.text.y = element_blank())

```

```{r demosupporthlmi2, echo=FALSE, fig.height=7.75, fig.keep='all',  warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Predicting support for developing high-level machine intelligence using demographic characteristics: results from a multiple linear regression that includes all demographic variables"}

# Generate the data for the graph
dev_md_support_hlmi_o <- lm_robust(formula = as.formula("Q17_clean ~ demo_age + demo_gender + 
               demo_white + demo_educ + demo_employ + 
    demo_pid3 + demo_income + demo_rel + demo_bornagain + demo_cs + 
    demo_prog"), data = d, weights = d$survey_weights)

dev_md_support_hlmi <- data.frame(variables = names(dev_md_support_hlmi_o$coefficients),
                     num = dev_md_support_hlmi_o$coefficients,
                     se = dev_md_support_hlmi_o$std.error,
                     p = dev_md_support_hlmi_o$p.value)
dev_md_support_hlmi$variables <- gsub(pattern = paste0(demo_var, collapse = "|"), replacement = "",
                         x = dev_md_support_hlmi$variables)
dev_md_support_hlmi$variables[dev_md_support_hlmi$variables == "Q5_clean"] <- 
  "Support for developing AI"

# # Make the stars
dev_md_support_hlmi$stars <- ""
dev_md_support_hlmi$stars[dev_md_support_hlmi$p < 0.05] <- "*"
dev_md_support_hlmi$stars[dev_md_support_hlmi$p < 0.01] <- "**"
dev_md_support_hlmi$stars[dev_md_support_hlmi$p < 0.001] <- "***"

# Make the text
dev_md_support_hlmi$new_text <- paste0(roundfunc(dev_md_support_hlmi$num), 
                      " (", roundfunc(dev_md_support_hlmi$se), 
                      ")", dev_md_support_hlmi$stars)


# add in the N
dev_md_support_hlmi_k <- rbind(data.frame(variables = dev_md_support_hlmi$variables, coef = dev_md_support_hlmi$new_text),
                  data.frame(variables = "\\textit{N} = 2000",
                             coef = paste0("\\textit{F}(", round(dev_md_support_hlmi_o$fstatistic[2]), ",",
                           round(dev_md_support_hlmi_o$fstatistic[3]), ") = ",
                           roundfunc(dev_md_support_hlmi_o$fstatistic[1]), "; \\textit{p}-value: <0.001")))


dev_md_support_hlmi$variables <- factor(dev_md_support_hlmi$variables, 
                           levels = rev(dev_md_support_hlmi$variables[c(2:nrow(dev_md_support_hlmi), 1)]))
# Make a graph
ggplot(data = dev_md_support_hlmi[dev_md_support_hlmi$variables != "(Intercept)",], 
       aes(x = variables, y = num, 
                                 ymin = qnorm(0.025) * se + num,
                                 ymax = qnorm(0.975) * se + num)) +
  geom_hline(yintercept = 0, linetype = 2, alpha = 0.5) +
  geom_pointrange(position = position_dodge(width = 0.9), size = 0.25) + 
  geom_text(aes(label = roundfunc(num)), 
            nudge_x = 0.4, alpha = 0.6) +
  coord_flip() + 
  scale_x_discrete(labels = function(x) str_wrap(x, width = 30),
                         name = "Demographic characteristics") + 
  scale_y_continuous(
    name = "Coefficient estimates (outcome standardized)") + 
  expand_limits(x = c(1, nrow(dev_md_support_hlmi))) +
  labs(
       caption = "Source: Center for the Governance of AI") + 
  theme_bw()

```

## The public expects high-level machine intelligence to be more harmful than good {#subsecharmgood}


This question sought to quantify respondents' expected outcome of high-level machine intelligence. (See [Appendix B](#expectedoutcome) for the question text.) Respondents were asked to consider the following:

> Suppose that high-level machine intelligence could be developed one day. How positive or negative do you expect the overall impact of high-level machine intelligence to be on humanity in the long run?

Americans, on average, expect that high-level machine intelligence will have a harmful impact on balance. Overall, 34% think that the technology will have a harmful impact; in particular, 12% said it could be extremely bad, leading to possible human extinction. More than a quarter of Americans think that high-level machine intelligence will be good for humanity, with 5% saying it will be extremely good. Since forecasting the impact of such technology on humanity is highly uncertain, 18% of respondents selected "I don't know." The correlation between one's expected outcome and one's support for developing high-level machine intelligence is 0.69. 

A similar question was asked to AI experts in @grace2018will; instead of merely selecting one expected outcome, the AI experts were asked to predict the likelihood of each outcome. In contrast to the general public, the expert respondents think that high-level machine intelligence will be more beneficial than harmful. [^probfn] Although they assign, on average, a 27% probability of high-level machine intelligence of being extremely good for humanity, they also assign, on average, a 9% probability of the technology being extremely bad, including possibly causing human extinction. 

[^probfn]: To make the two groups' results more comparable, we calculated the expected value of the experts' predicted outcomes so that it is on the same -2 to 2 scale as the public's responses. To calculate this expected value, we averaged the sums of each expert's predicted likelihoods multiplied by the corresponding outcomes; we used the same numerical outcome as described in the previous subsection. The expected value of the experts' predicted outcomes is 0.08, contrasted with the public's average response of -0.17. 

```{r expectedoutcome, echo=FALSE, results='hide', fig.height=4.5, fig.keep='all', warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Expected positive or negative impact of high-level machine intelligence on humanity"}

# Get the overall mean for recoding missing data
outcome_hlmi_overall_mean <- wtd.mean(relabel_var(d$Q18,
                                      c(1:6, 8, 9),
                                      c(2, 1, 0, -1, -2, NA, NA, NA)),
                                      weights = d$survey_weights, na.rm = TRUE)

# Prepare the data
outcome_hlmi_value_table <- catvar_func(
    outcome = 
      "Expected positive or negative impact of high-level machine intelligence on humanity",
    outcome_var = d$Q18,
    label_var = d0$Q18,
    output_type = "value_table",
    shown = rep(TRUE, nrow(d)),
    num_missing = 8,
    num_DK = 6, missing_recode = outcome_hlmi_overall_mean,
    new_values <- c(2, 1, 0, -1, -2, 98, 99, 100))
outcome_hlmi_value_table$num <- support_hlmi_value_table$new_values
outcome_hlmi_value_sum <- catvar_func(
    outcome = 
      "Expected positive or negative impact of high-level machine intelligence on humanity",
    outcome_var = d$Q18,
    label_var = d0$Q18,
    output_type = "num_value",
    shown = rep(TRUE, nrow(d)),
    num_missing = 8,
    num_DK = 6, missing_recode = outcome_hlmi_overall_mean,
    new_values <- c(2, 1, 0, -1, -2, NA, NA, NA))
outcome_hlmi_value_table$Prop_text <- 
  round(outcome_hlmi_value_table$Prop*100, digits = 0)
outcome_hlmi_value_table$Prop_text[outcome_hlmi_value_table$Prop_text == "0"] <- "<1"
# Make the graph
outcome_hlmi_value_table$labels <- factor(outcome_hlmi_value_table$labels, 
                                          levels = outcome_hlmi_value_table$labels[c(5:1, 6:7)])
ggplot() +
  geom_bar(data = outcome_hlmi_value_table[outcome_hlmi_value_table$Prop !=0,],
           aes(x = num, y = Prop, fill = labels), stat = "identity", 
           alpha = 0.6, color = "black") +
  geom_errorbar(data = 
                  outcome_hlmi_value_table[outcome_hlmi_value_table$Prop !=0,], 
                aes(x = num, ymin = Prop + qnorm(0.025)*se,
                    ymax = Prop + qnorm(0.975)*se), width = 0.1) +
  geom_text(data = outcome_hlmi_value_table, aes(x = num, 
                                           label = Prop_text), 
            y = 0.02, nudge_x = 0.25) +
  scale_x_continuous(breaks = outcome_hlmi_value_table$num[order(outcome_hlmi_value_table$num)],
    labels = str_wrap(outcome_hlmi_value_table$labels[order(outcome_hlmi_value_table$num)], 
                      width = 10)) +
  facet_grid(~group, scales = "free_x", space = "free_x") + theme_bw() +
  geom_text(data = outcome_hlmi_value_sum, aes(x = 0, label = sum_stat,
                                       y = max(outcome_hlmi_value_table$Prop)+0.05)) +
  scale_y_continuous(labels = scales::percent, 
                     limits = c(0, max(outcome_hlmi_value_table$Prop)+0.05)) +
  xlab("Responses") + ylab("Percentage of respondents") + 
    scale_fill_manual(values = c("#a6611a", "#dfc27d", 
                                 "#f5f5f5", "#80cdc1", "#018571",
                               "grey65", "grey65")) + 
  labs(
       caption = "Source: Center for the Governance of AI") + theme(legend.position = "none")

d$Q18_clean <- relabel_var(d$Q18, c(1:6, 8, 9),
                           c(2, 1, 0, -1, -2, NA, NA, NA))
d$Q18_clean[is.na(d$Q18_clean)] <- outcome_hlmi_overall_mean
cor(d$Q17_clean, d$Q18_clean)

```

\newpage

\addtocontents{toc}{\protect\setcounter{tocdepth}{1}}

# (APPENDIX) Appendices {-}

\counterwithin{figure}{section}
\counterwithin{table}{section}

# Appendix A: Methodology {#appmethod}

## YouGov sampling and weights {#yougovsampling}

YouGov interviewed 2,387 respondents who were then matched down to a sample of 2,000 to produce the final dataset. The respondents were matched to a sampling frame on gender, age, race, and education. The frame was constructed by stratified sampling from the full 2016 American Community Survey (ACS) one-year sample with selection within strata by weighted sampling with replacements (using the person weights on the public use file).

The matched cases were weighted to the sampling frame using propensity scores. The matched cases and the frame were combined and a logistic regression was estimated for inclusion in the frame. The propensity score function included age, gender, race/ethnicity, years of education, and geographic region. The propensity scores were grouped into deciles of the estimated propensity score in the frame and post-stratified according to these deciles.

The weights were then post-stratified on 2016 U.S. presidential vote choice, and a four-way stratification of gender, age (four-categories), race (four-categories), and education (four-categories), to produce the final weight.

## Demographic subgroups {#appdemosubgroups}

We use the following demographic subjects in our analysis:

- Age group as defined by [Pew Research Center](http://www.pewresearch.org/fact-tank/2018/03/01/defining-generations-where-millennials-end-and-post-millennials-begin/): Millennial/post-Millennial adults (born after 1980; ages 18-37 in 2018), Gen Xers (born 1965-1980; ages 38-53 in 2018), Baby Boomers (born 1946-1964; ages 54-72 in 2018), Silents/Greatest Generation (1945 and earlier; ages 73 and over in 2018)
- Gender: male, female
- Race: white, non-white
- Level of education: graduated from high school or less, some college (including two-year college), graduated from a four-year college or more
- Employment status: employed (full- or part-time), not employed
- Annual household income: less than \$30,000, \$30,000-70,000, \$70,000-100,000, more than \$100,000, prefer not to say
- Political party identification: Democrats (includes those who lean Democrat), Republicans (includes those who lean Republican), Independents/Others
- Religion: Christian, follow other religions, non-religious
- Identifies as a born-again Christian: yes, no
- Completed a computer science or engineering degree in undergraduate or graduate school: yes, no
- Has computer science or programming experience: yes, no

We report the unweighted sample sizes of the demographic subgroups in Table \@ref(tab:usesfreq).

```{r usesfreq, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE}
demo_count <- function(varname) {
  as.data.frame(table(d[,varname]))
}

# Generate the table
demo_freq <- do.call(rbind, lapply(X = demo_var, demo_count))

kable(x = demo_freq, 
      format = "pandoc",
                   caption = "Size of demographic subgroups",
        col.names = c("Demographic subgroups", "Unweighted sample sizes"))

```

## Analysis {#appanalysis}

We [pre-registered the analysis](https://osf.io/7gqvm/) of this survey on Open Science Framework. Pre-registration increases research transparency by requiring researchers to specify their analysis before analyzing the data [@nosek2018preregistration]. Doing so prevents researchers from misusing data analysis to come up with statistically significant results when they do not exist, otherwise known as $p$-hacking. 

Unless otherwise specified, we performed the following procedure:

- Survey weights provided by YouGov were used in our primary analysis. For transparency, Appendix B contains the unweighted topline results, including raw frequencies.

- For estimates of summary statistics or coefficients, "don't know" or missing responses were re-coded to the weighted overall mean, unconditional on treatment conditions. Almost all questions had a "don't know" option. If more than 10% of the variable's values were don't know" or missing, we included a (standardized) dummy variable for "don't know"/missing in the analysis. For survey experiment questions, we compared "don't know"/missing rates across experimental conditions. Our decision was informed by the [Standard Operating Procedures for Don Green's Lab at Columbia University](https://github.com/acoppock/Green-Lab-SOP) [@lin2016standard].

- Heteroscedasticity-consistent standard errors were used to generate the margins of error at the 95% confidence level. We report cluster-robust standard errors whenever there is clustering by respondent. In figures, each error bar shows the 95% confidence intervals. Each confidence ellipse shows the 95% confidence region of the bivariate means assuming the two variables are distributed multivariate normal. 

- In regression tables, * denotes $p<0.05$, ** denotes $p<0.01$, and *** denotes $p<0.001$. 

## Data sharing {#datasharing}

We plan to make our survey data, as well as the R and Markdown code that produced this report, publicly available through the Harvard Dataverse six months after the publication of this report.

\newpage

# Appendix B: Topline questionnaire {#apptopline}

Below, we present the survey text as shown to respondents. The numerical codings are shown in parentheses following each answer choice. 

In addition, we report the topline results: percentages weighted to be representative of the U.S. adult population, the unweighted raw percentages, and the raw frequencies. Note that in all survey experiments, respondents were randomly assigned to each experimental group with equal probability. 

## Global risks {#global_risks}

```{r globalriskq, echo=FALSE, fig.height=5, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
# Remove all the big datasets
rm("all_hlmi")
rm("md_Q15")
rm("expert_lines")
rm("support_diff")
rm("curves.cs")
rm("pdata.cs")
rm("bs_res")
rm("bs_res.cs")
rm("expert_bs")

# Function to generate the tables
global_risk_q1 <- function(risk_num) {
  res_w <- catvar_func(
  outcome = label(d[,paste0("Q1_", risk_num)]),
  outcome_var = d[,paste0("Q1_", risk_num)],
  label_var = d0[,paste0("Q1_", risk_num)],
  output_type = "value_table",
  shown = d0[,paste0("Q1_", risk_num)] != 99,
  num_missing = 98,
  num_DK = 8, edit_labels = FALSE,
  new_values <- c(1:7, NA, NA, NA),
  survey_weights = d$survey_weights, 
  missing_recode = 0
  ) 
  res_u <- catvar_func(
  outcome = label(d[,paste0("Q1_", risk_num)]),
  outcome_var = d[,paste0("Q1_", risk_num)],
  label_var = d0[,paste0("Q1_", risk_num)],
  output_type = "value_table",
  shown = d0[,paste0("Q1_", risk_num)] != 99,
  num_missing = 98,
  num_DK = 8, edit_labels = FALSE,
  new_values <- c(1:7, NA, NA, NA),
  survey_weights = d$weight1, 
  missing_recode = 0
  ) 
  cat_table <- data.frame(responses = res_w$labels, 
                          percent_w = roundfunc(res_w$Prop *100, 2),
             percent_u = roundfunc(res_u$Prop *100, 2), freq_u = res_u$Freq)
  kable(x = cat_table, format = "pandoc",
                   caption = paste0("Likelihood - ", global_risks[risk_num],
                                    "; \\textit{N} = ", 
                                    sum(d0[,paste0("Q1_", risk_num)] != 99)), 
        col.names = c("Responses", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies"))
}

global_risk_q2 <- function(risk_num) {
  res_w <- catvar_func(
  outcome = label(d[,paste0("Q2_", risk_num)]),
  outcome_var = d[,paste0("Q2_", risk_num)],
  label_var = d0[,paste0("Q2_", risk_num)],
  output_type = "value_table",
  shown = d0[,paste0("Q2_", risk_num)] != 9,
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(1:5, NA, NA, NA),
  survey_weights = d$survey_weights, 
  missing_recode = 0
  ) 
  res_u <- catvar_func(
  outcome = label(d[,paste0("Q2_", risk_num)]),
  outcome_var = d[,paste0("Q2_", risk_num)],
  label_var = d0[,paste0("Q2_", risk_num)],
  output_type = "value_table",
  shown = d0[,paste0("Q2_", risk_num)] != 9,
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(1:5, NA, NA, NA),
  survey_weights = d$weight1, 
  missing_recode = 0
  ) 
  cat_table <- data.frame(responses = res_w$labels, 
                          percent_w = roundfunc(res_w$Prop *100, 2),
             percent_u = roundfunc(res_u$Prop *100, 2), freq_u = res_u$Freq)
  kable(x = cat_table, format = "pandoc",
                   caption = paste0("Size of negative impact - ",
                                    global_risks[risk_num],
                                    "; \\textit{N} = ", 
                                    sum(d0[,paste0("Q2_", risk_num)] != 9)), 
        col.names = c("Responses", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies"))
}

gen_topline <- function(group_var, varname, agg_func) {
  # Frequency table
  w_tab <- do.call(rbind, lapply(X = 1:length(group_var), agg_func,
                                 surveyweights = d$survey_weights,
                                 output_type = "value_table"))
  u_tab <- do.call(rbind, lapply(X = 1:length(group_var), agg_func,
                                 surveyweights = d$weight1,
                                 output_type = "value_table"))
  
  d_tab <- data.frame(responses = w_tab$labels,
                      group = w_tab$outcome,
                      perc_w = roundfunc(w_tab$Prop*100, 2),
                      perc_u = roundfunc(u_tab$Prop*100, 2),
                      freq = u_tab$Freq, 
                      n_shown = w_tab$n_show)
  
  output_func <- function(group) {
    kable(
      x = d_tab[d_tab$group == group,
                c("responses", "perc_w", "perc_u", "freq")],
      format = "pandoc",
      caption = paste0(varname, " - ", group, "; \\textit{N} = ",
                       mean(d_tab$n_shown[d_tab$group == group])),
      col.names = c(
        "Answer choices",
        "Percentages (weighted)",
        "Percentages (unweighted)",
        "Raw frequencies"
      ),
      row.names = FALSE
    )
  }
  
  return(lapply(group_var, output_func))
}

```

[All respondents were presented with the following prompt.]

We want to get your opinion about global risks. A "global risk" is an uncertain event or condition that, if it happens, could cause a significant negative impact for at least 10 percent of the world’s population. That is at least 1 in 10 people around the world could experience a significant negative impact. 

You will be asked to consider 5 potential global risks. 

[Respondents were presented with five items randomly selected from the list below. One item was shown at a time.]

- **Failure to address climate change**: Continued failure of governments and businesses to pass effective measures to reduce climate change, protect people, and help those impacted by climate change to adapt. 
- **Failure of regional or global governance**: Regional organizations (e.g., the European Union) or global organizations (e.g., the United Nations) are unable to resolve issues of economic, political, or environmental importance. 
- **Conflict between major countries**: Disputes between major countries that lead to economic, military, cyber, or societal conflicts. 
- **Weapons of mass destruction**: Use of nuclear, chemical, biological or radiological weapons, creating international crises and killing large numbers of people. 
- **Large-scale involuntary migration**: Large-scale involuntary movement of people, such as refugees, caused by conflict, disasters, environmental or economic reasons.
- **Rapid and massive spread of infectious diseases**: The uncontrolled spread of infectious diseases, for instance as a result of resistance to antibiotics, that leads to widespread deaths and economic disruptions. 
- **Water crises**: A large decline in the available quality and quantity of fresh water that harms human health and economic activity. 
- **Food crises**: Large numbers of people are unable to buy or access food. 
Harmful consequences of artificial intelligence (AI): Intended or unintended consequences of artificial intelligence that causes widespread harm to humans, the economy, and the environment.
- **Harmful consequences of synthetic biology**: Intended or unintended consequences of synthetic biology, such as genetic engineering, that causes widespread harm to humans, the economy, and the environment. 
- **Large-scale cyber attacks**: Large-scale cyber attacks that cause large economic damages, tensions between countries, and widespread loss of trust in the internet.
- **Large-scale terrorist attacks**: Individuals or non-government groups with political or religious goals that cause large numbers of deaths and major material damage. 
- **Global recession**: Economic decline in several major countries that leads to a decrease in income and high unemployment. 
- **Extreme weather events**: Extreme weather events that cause large numbers of deaths as well as damage to property, infrastructure, and the environment. 
- **Major natural disasters**: Earthquakes, volcanic activity, landslides, tsunamis, or geomagnetic storms that cause large numbers of deaths as well as damage to property, infrastructure, and the environment. 

QUESTION: 

**What is the likelihood of [INSERT GLOBAL RISK] happening globally within the next 10 years?** Please use the slider to indicate your answer. 0% chance means it will certainly not happen and 100% chance means it will certainly happen.

ANSWER CHOICES [^probexplain]:

- Very unlikely: less than 5% chance (2.5%)
- Unlikely: 5-20% chance (12.5%)
- Somewhat unlikely: 20-40% chance (30%)
- Equally likely as unlikely: 40-60% chance (50%)
- Somewhat likely: 60-80% chance (70%)
- Likely: 80-95% chance (87.5%)
- Very likely: more than 95% chance (97.5%)
- I don’t know

[^probexplain]: For this and other questions that ask respondents about likelihoods, each multiple-choice answer was coded to the mean value across the probabilities in the answer's range. 

QUESTION: 

If [INSERT GLOBAL RISK] were to happen, what would be the size of the negative impact for several countries or industries within the next 10 years?

ANSWER CHOICES:

- Minimal (0)
- Minor (1)
- Moderate (2)
- Severe (3)
- Catastrophic (4)
- I don’t know

```{r globalriskq1i1, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(1)
```

```{r globalriskq1i2, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(2)
```

```{r globalriskq1i3, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(3)
```

```{r globalriskq1i4, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(4)
```

```{r globalriskq1i5, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(5)
```

```{r globalriskq1i6, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(6)
```

```{r globalriskq1i7, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(7)
```

```{r globalriskq1i8, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(8)
```

```{r globalriskq1i9, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(9)
```

```{r globalriskq1i10, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(10)
```

```{r globalriskq1i11, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(11)
```

```{r globalriskq1i12, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(12)
```

```{r globalriskq1i13, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(13)
```

```{r globalriskq1i14, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(14)
```

```{r globalriskq1i15, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q1(15)
```

```{r globalriskq2i1, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(1)
```

```{r globalriskq2i2, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(2)
```

```{r globalriskq2i3, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(3)
```

```{r globalriskq2i4, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(4)
```

```{r globalriskq2i5, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(5)
```

```{r globalriskq2i6, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(6)
```

```{r globalriskq2i7, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(7)
```

```{r globalriskq2i8, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(8)
```

```{r globalriskq2i9, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(9)
```

```{r globalriskq2i10, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(10)
```

```{r globalriskq2i11, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(11)
```

```{r globalriskq2i12, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(12)
```

```{r globalriskq2i13, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(13)
```

```{r globalriskq2i14, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(14)
```

```{r globalriskq2i15, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
global_risk_q2(15)
```

## Survey experiment: what the public considers AI, automation, machine learning, and robotics {#considersai}

[Respondents were randomly assigned to one of the four questions. The order of answer choices was randomized, except that “None of the above” was always shown last.] 

QUESTIONS:

- In your opinion, which of the following technologies, if any, uses **artificial intelligence** (AI)? Select all the apply. 
- In your opinion, which of the following technologies, if any, uses **automation**?  Select all that apply.
- In your opinion, which of the following technologies, if any, uses **machine learning**? Select all that apply. 
- In your opinion, which of the following technologies, if any, uses **robotics**? Select all that apply. 

ANSWER CHOICES: 

- Virtual assistants (e.g., Siri, Google Assistant, Amazon Alexa)
- Smart speakers (e.g., Amazon Echo, Google Home, Apple Homepod)
- Facebook photo tagging
- Google Search
- Recommendations for Netflix movies or Amazon ebooks 
- Google Translate
- Driverless cars and trucks
- Social robots that can interact with humans 
- Industrial robots used in manufacturing
- Drones that do not require a human controller 
- None of the above

```{r usesai, echo=FALSE, fig.height=5, fig.keep='all'}
# Run the analysis 
whatsai_d <- data.frame(techtreat = d$q3new_treat,
                        survey_weights = d$survey_weights,
                        do.call(cbind, lapply(1:10, whatsai)))

whatsai_d <- reshape2::melt(whatsai_d, id = c("techtreat", "survey_weights")) %>% 
  group_by(techtreat, variable) %>% dplyr::summarise(
    prop_w = 
      roundfunc(100*md_weight(value, weights = survey_weights, which_stat = "mean"), 2),
    prop_u = roundfunc(100*mean(value), 2),
    raw_freq = sum(value),
    N = n())

# Clean up
whatsai_d$techtreat <- relabel_var(old_var = whatsai_d$techtreat, 
                                   old_labels = c(1:4),
            new_labels = c("Artificial intelligence (AI)", 
                           "Automation", "Machine learning", "Robotics"))

whatsai_d$variable <- relabel_var(old_var = whatsai_d$variable, 
                                  old_labels = levels(whatsai_d$variable),
            new_labels = label(d0)[paste0("Q3new_", 1:10)]) %>% 
  factor(levels = rev(label(d0)[paste0("Q3new_", 1:10)]))

# Generate the kables
whatsai_table <- function(tech) {
  kable(x = whatsai_d[whatsai_d$techtreat == tech, 2:5], 
      format = "latex", booktabs = TRUE,
                   caption = paste0(tech,
                                    "; \\textit{N} = ",
     mean(whatsai_d$N[whatsai_d$techtreat == tech])),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies"),
     longtable = TRUE) 
}
```

```{r usesai1, echo=FALSE, fig.height=5, fig.keep='all', results='asis'}
whatsai_table(unique(whatsai_d$techtreat)[1])  %>%
  column_spec(1, width = "4cm") 
```

```{r usesai2, echo=FALSE, fig.height=5, fig.keep='all', results='asis'}
whatsai_table(unique(whatsai_d$techtreat)[2])  %>%
  column_spec(1, width = "4cm") 
```

```{r usesai3, echo=FALSE, fig.height=5, fig.keep='all', results='asis'}
whatsai_table(unique(whatsai_d$techtreat)[3])  %>%
  column_spec(1, width = "4cm") 
```

```{r usesai4, echo=FALSE, fig.height=5, fig.keep='all', results='asis'}
whatsai_table(unique(whatsai_d$techtreat)[4])  %>%
  column_spec(1, width = "4cm") 
```

## Knowledge of computer science (CS)/technology

QUESTION: 

What is your knowledge of computer science/technology? (Select all that apply.) 

ANSWER CHOICES:

- I have taken at least one college-level course in computer science.
- I have a computer science or engineering undergraduate degree.
- I have a graduate degree in computer science or engineering.
- I have programming experience.
- I don't have any of the educational or work experiences described above.

```{r techbg, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
# Helper function 
techbackground <- function(technum, output_type = "num_outcome") {
  catvar_func(
  outcome = label(d0[,paste0("Q4_", technum)]),
  outcome_var = d[,paste0("Q4_", technum)],
  label_var = d0[,paste0("Q4_", technum)],
  output_type = output_type,
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = -99, 
  new_values <- c(1, 0, NA, NA), 
  survey_weights = d$survey_weights, 
  missing_recode = wtd.mean(relabel_var(d[,paste0("Q4_", technum)],
                                        c(1, 2, 8, 9), c(1, 2, NA, NA)), 
                            weights = d[,"survey_weights"], na.rm = TRUE)
  )  
}

techbg_d <- data.frame(id = d$caseid,
                        survey_weights = d$survey_weights,
                        do.call(cbind, lapply(1:5, techbackground)))

techbg_d <- reshape2::melt(techbg_d, id = c("id", "survey_weights")) %>% 
  group_by(variable) %>% dplyr::summarise(
    prop_w = 
      roundfunc(100*md_weight(value, weights = survey_weights, which_stat = "mean"), 2),
    prop_u = roundfunc(100*mean(value), 2),
    raw_freq = sum(value),
    N = n())

# Clean up
techbg_label <- c("Took at least one college-level course in CS",
"CS or engineering undergraduate degree", 
"CS or engineering graduate degree",
"Have programming experience", 
"None of the above")

techbg_d$variable <- relabel_var(old_var = techbg_d$variable, 
                                  old_labels = levels(techbg_d$variable),
            new_labels = techbg_label) %>% 
  factor(levels = techbg_label)

kable(x = techbg_d[1:4], 
      format = "pandoc",
                   caption = paste0("Computer science/technology background; \\textit{N} = ",
     mean(techbg_d$N)),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies")) 

```

## Support for developing AI {#supportdevai}

[All respondents were presented with the following prompt.]

Next, we would like to ask you questions about your attitudes toward artificial intelligence. 

Artificial Intelligence (AI) refers to computer systems that perform tasks or make decisions that usually require human intelligence. AI can perform these tasks or make these decisions without explicit human instructions. Today, AI has been used in the following applications:

[Respondents were shown five items randomly selected from the list below.]

- Translate over 100 different languages
- Predict one’s Google searches
- Identify people from their photos
- Diagnose diseases like skin cancer and common illnesses
- Predict who are at risk of various diseases 
- Help run factories and warehouses
- Block spam email
- Play computer games
- Help conduct legal case research
- Categorize photos and videos
- Detect plagiarism in essays
- Spot abusive messages on social media 
- Predict what one is likely to buy online
- Predict what movies or TV shows one is likely to watch online 

QUESTION: 

**How much do you support or oppose the development of AI?** 

ANSWER CHOICES:

- Strongly support (2)
- Somewhat support (1)
- Neither support nor oppose (0)
- Somewhat oppose (-1)
- Strongly oppose (-2)
- I don’t know

```{r supportaitable, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE}

supportdev_w <- catvar_func(
  outcome = label(d0$Q5),
  outcome_var = d$Q5,
  label_var = d0$Q5,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(2, 1, 0, -1, -2, NA, NA, NA),
  survey_weights = d$survey_weights,
  missing_recode = dev_ai_overall_mean
  )

supportdev_u <- catvar_func(
  outcome = label(d0$Q5),
  outcome_var = d$Q5,
  label_var = d0$Q5,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(2, 1, 0, -1, -2, NA, NA, NA),
  survey_weights = d$weight1,
  missing_recode = dev_ai_overall_mean
  )

supportdev_d <- data.frame(labels = supportdev_w$labels, 
                           w_perc = roundfunc(100*supportdev_w$Prop, 2),
                           u_perc = roundfunc(100*supportdev_u$Prop, 2),
                           freq = supportdev_u$Freq)

kable(x = supportdev_d, 
      format = "pandoc",
                   caption = paste0("Support for developing AI; \\textit{N} = 2000"),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies")) 

```

## Survey experiment: AI and/or robots should be carefully managed {#manageexp}

QUESTION:

Please tell me to what extent you agree or disagree with the following statement.

[Respondents were presented with one statement randomly selected from the list below.]

- AI and robots are technologies that require careful management.
- AI is a technology that requires careful management.
- Robots are technologies that require careful management.

ANSWER CHOICES:

- Totally agree (2)
- Tend to agree (1)
- Tend to disagree (-1)
- Totally disagree (-2)
- I don’t know

```{r aistatementtable, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
manage_exp <- c("AI and robots", "AI", "Robots")

ai_statement_func <- function(exp_group, output_type, surveyweights) {
  data.frame(catvar_func(outcome = manage_exp[exp_group], 
              outcome_var = d[,"Q5b"], 
              shown = d$q5b_treat == exp_group,
              label_var = d0[,"Q5b"], 
              num_missing = 8, num_DK = 5,
              new_values = c(c(2, 1, -1, 2), NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d$q5b_treat == exp_group))
  
}

# Frequency tables
manage_exp_res <- gen_topline(group_var = manage_exp, 
            varname = "Responses to statement", 
            agg_func = ai_statement_func)

```

```{r aistatementtable1, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
manage_exp_res[[1]]
```

```{r aistatementtable2, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
manage_exp_res[[2]]
```

```{r aistatementtable3, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
manage_exp_res[[3]]
```

## Trust of actors to develop AI {#trustdevai}

QUESTION: 

**How much confidence, if any, do you have in each of the following to develop AI in the best interests of the public?** 

[Respondents were shown five items randomly selected from the list below. We included explainer text for actors not well known to the public; respondents could view the explainer text by hovering their mouse over the actor's name. The items and the answer choices were shown in a matrix format.]

- The U.S. military
- The U.S. civilian government
- National Security Agency (NSA)
- Federal Bureau of Investigation (FBI)
- Central Intelligence Agency (CIA)
- North Atlantic Treaty Organization (NATO)
  - Explainer text for NATO: NATO is a military alliance that includes 28 countries including most of Europe, as well as the U.S. and Canada. 
- An international research organization (e.g., CERN)
  - Explainer text for CERN: The European Organization for Nuclear Research, known as CERN, is a European research organization that operates the largest particle physics laboratory in the world.
- Tech companies
- Google 
- Facebook 
- Apple 
- Microsoft 
- Amazon
- A non-profit AI research organization (e.g., OpenAI)
  - Explainer text for OpenAI: Open AI is an AI non-profit organization with backing from tech investors that seeks to develop safe AI.
University researchers

ANSWER CHOICES:

- A great deal of confidence (3)
- A fair amount of confidence (2)
- Not too much confidence (1)
- No confidence (0)
- I don’t know

```{r trusttabledev, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
# Function to generate the tables
# Helper function
ai_dev_func2 <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = dev_actors[variable_number], 
              outcome_var = d[,paste0("Q6_", variable_number)], 
              shown = d0[,paste0("Q6_org_", variable_number)] == 1,
              label_var = d0[,paste0("Q6_", variable_number)], 
              num_missing = 8, num_DK = 5,
              new_values = c(3, 2, 1, 0, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = ai_dev_overall_mean,
              output_type), 
             n_show = sum(shown = d0[,paste0("Q6_org_", variable_number)] == 1))
  
}

# Frequency table
trust_dev_table_w <- do.call(rbind, lapply(X = 1:15, ai_dev_func2,
                                         surveyweights = d$survey_weights,
                                   output_type = "value_table"))

trust_dev_table_u <- do.call(rbind, lapply(X = 1:15, ai_dev_func2,
                                         surveyweights = d$weight1,
                                   output_type = "value_table"))

trust_dev_t_d <- data.frame(responses = trust_dev_table_w$labels,
                          actors = trust_dev_table_w$outcome,
                          perc_w = roundfunc(trust_dev_table_w$Prop*100, 2),
                          perc_u = roundfunc(trust_dev_table_u$Prop*100, 2),
                          freq = trust_dev_table_u$Freq, 
                          n_shown = trust_dev_table_u$n_show)

trust_dev_t_func <- function(actor) {
  kable(x = trust_dev_t_d[trust_dev_t_d$actors == actor, 
                          c("responses", "perc_w", "perc_u", "freq")], 
      format = "pandoc",
      caption = paste0(actor, "; \\textit{N} = ",
     mean(trust_dev_t_d$n_shown[trust_dev_t_d$actors == actor])),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies"), row.names = FALSE) 
}

trust_dev_t_d_actors <- unique(trust_dev_t_d$actors)

```


```{r trustdevi1, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[1])
```

```{r trustdevi2, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[2])
```

```{r trustdevi3, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[3])
```

```{r trustdevi4, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[4])
```

```{r trustdevi5, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[5])
```

```{r trustdevi6, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[6])
```

```{r trustdevi7, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[7])
```

```{r trustdevi8, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[8])
```

```{r trustdevi9, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[9])
```

```{r trustdevi10, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[10])
```

```{r trustdevi11, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[11])
```

```{r trustdevi12, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[12])
```

```{r trustdevi13, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[13])
```

```{r trustdevi14, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[14])
```

```{r trustdevi15, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_dev_t_func(trust_dev_t_d_actors[15])
```

## Trust of actors to manage AI {#trustmanageai}

QUESTION: 

**How much confidence, if any, do you have in each of the following to manage the development and use of AI in the best interests of the public?**

[Respondents were shown five items randomly selected from the list below. We included explainer text for actors not well known to the public; respondents could view the explainer text by hovering their mouse over the actor's name. The items and the answer choices were shown in a matrix format.]

- U.S. federal government 
- U.S. state governments
- International organizations (e.g., United Nations, European Union)
- The United Nations (UN)
- An intergovernmental research organization (e.g., CERN)
  - Explainer text for CERN: The European Organization for Nuclear Research, known as CERN, is a European research organization that operates the largest particle physics laboratory in the world.
- Tech companies 
- Google 
- Facebook 
- Apple 
- Microsoft 
- Amazon
- Non-government scientific organizations (e.g., AAAI)
  - Explainer text for AAAI: Association for the Advancement of Artificial Intelligence (AAAI) is a non-government scientific organization that promotes research in, and responsible use of AI.
- Partnership on AI, an association of tech companies, academics, and civil society groups

ANSWER CHOICES:

- A great deal of confidence (3)
- A fair amount of confidence (2)
- Not too much confidence (1)
- No confidence (0)
- I don’t know


```{r trusttable2, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}

# Helper function
ai_manage_func2 <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = manage_actors[variable_number], 
              outcome_var = d[,paste0("Q7_", variable_number)], 
              shown = d[,paste0("Q7_org_", variable_number)] == 1,
              label_var = d0[,paste0("Q7_", variable_number)], 
              num_missing = 8, num_DK = 5,
              new_values = c(3, 2, 1, 0, NA, NA, NA),  
              missing_recode = ai_manage_overall_mean,
              survey_weights = surveyweights, edit_labels = FALSE,
              output_type), 
             n_shown = sum(d[,paste0("Q7_org_", variable_number)] == 1))
}

# Frequency table
trust_manage_table_w <- do.call(rbind, lapply(X = 1:13, ai_manage_func2,
                                         surveyweights = d$survey_weights,
                                   output_type = "value_table"))

trust_manage_table_u <- do.call(rbind, lapply(X = 1:13, ai_manage_func2,
                                         surveyweights = d$weight1,
                                   output_type = "value_table"))

trust_manage_t_d <- data.frame(responses = trust_manage_table_w$labels,
                          actors = trust_manage_table_w$outcome,
                          perc_w = roundfunc(trust_manage_table_w$Prop*100, 2),
                          perc_u = roundfunc(trust_manage_table_u$Prop*100, 2),
                          freq = trust_manage_table_u$Freq, 
                          n_shown = trust_manage_table_u$n_show)

trust_manage_t_func <- function(actor) {
  kable(x = trust_manage_t_d[trust_manage_t_d$actors == actor, 
                          c("responses", "perc_w", "perc_u", "freq")], 
      format = "pandoc",
      caption = paste0(actor, "; \\textit{N} = ",
     mean(trust_manage_t_d$n_shown[trust_manage_t_d$actors == actor])),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies"), row.names = FALSE) 
}


trust_manage_t_actors <- unique(trust_manage_t_d$actors)

```

```{r trustmanagei1, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[1])
```

```{r trustmanagei2, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[2])
```

```{r trustmanagei3, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[3])
```

```{r trustmanagei4, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[4])
```

```{r trustmanagei5, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[5])
```

```{r trustmanagei6, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[6])
```

```{r trustmanagei7, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[7])
```

```{r trustmanagei8, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[8])
```

```{r trustmanagei9, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[9])
```

```{r trustmanagei10, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[10])
```

```{r trustmanagei11, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[11])
```

```{r trustmanagei12, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[12])
```

```{r trustmanagei13, echo=FALSE, fig.keep='all', warning=FALSE, cache=TRUE, results='asis'}
trust_manage_t_func(trust_manage_t_actors[13])
```

## AI governance challenges {#govchallenges}

We would like you to consider some potential policy issues related to AI. Please consider the following: 

[Respondents were shown five randomly-selected items from the list below, one item at a time. For ease of comprehension, we include the shorten labels used in the figures in square brackets.]

- **[Hiring bias] Fairness and transparency in AI used in hiring**: Increasingly, employers are using AI to make hiring decisions. AI has the potential to make less biased hiring decisions than humans. But algorithms trained on biased data can lead to lead to hiring practices that discriminate against certain groups. Also, AI used in this application may lack transparency, such that human users do not understand what the algorithm is doing, or why it reaches certain decisions in specific cases.
- **[Criminal justice bias] Fairness and transparency in AI used in criminal justice**: Increasingly, the criminal justice system is using AI to make sentencing and parole decisions. AI has the potential to make less biased hiring decisions than humans. But algorithms trained on biased data could lead to discrimination against certain groups. Also, AI used in this application may lack transparency such that human users do not understand what the algorithm is doing, or why it reaches certain decisions in specific cases.
- **[Disease diagnosis] Accuracy and transparency in AI used for disease diagnosis**: Increasingly, AI software has been used to diagnose diseases, such as heart disease and cancer. One challenge is to make sure the AI can correctly diagnose those who have the disease and not mistakenly diagnose those who do not have the disease. Another challenge is that AI used in this application may lack transparency such that human users do not understand what the algorithm is doing, or why it reaches certain decisions in specific cases.
- **[Data privacy] Protect data privacy**: Algorithms used in AI applications are often trained on vast amounts of personal data, including medical records, social media content, and financial transactions. Some worry that data used to train algorithms are not collected, used, and stored in ways that protect personal privacy.
- **[Autonomous vehicles] Make sure autonomous vehicles are safe**: Companies are developing self-driving cars and trucks that require little or no input from humans. Some worry about the safety of autonomous vehicles for those riding in them as well as for other vehicles, cyclists, and pedestrians. 
- **[Ditigal manipulation] Prevent AI from being used to spread fake and harmful content online**: AI has been used by governments, private groups, and individuals to harm or manipulate internet users. For instance, automated bots have been used to generate and spread false and/or harmful news stories, audios, and videos. 
- **[Cyber attacks] Prevent AI cyber attacks against governments, companies, organizations, and individuals**: Computer scientists have shown that AI can be used to launch effective cyber attacks. AI could be used to hack into servers to steal sensitive information, shut down critical infrastructures like power grids or hospital networks, or scale up targeted phishing attacks. 
- **[Surveillance] Prevent AI-assisted surveillance from violating privacy and civil liberties**: AI can be used to process and analyze large amounts of text, photo, audio, and video data from social media, mobile communications, and CCTV cameras. Some worry that governments, companies, and employers could use AI to increase their surveillance capabilities.
- **[U.S.-China arms race] Prevent escalation of a U.S.-China AI arms race**: Leading analysts believe that an AI arms race is beginning, in which the U.S. and China are investing billions of dollars to develop powerful AI systems for surveillance, autonomous weapons, cyber operations, propaganda, and command and control systems. Some worry that a U.S.-China arms race could lead to extreme dangers. To stay ahead, the U.S. and China may race to deploy advanced military AI systems that they do not fully understand or can control. We could see catastrophic accidents, such as a rapid, automated escalation involving cyber and nuclear weapons.
- **[Value alignment] Make sure AI systems are safe, trustworthy, and aligned with human values**: As AI systems become more advanced, they will increasingly make decisions without human input. One potential fear is that AI systems, while performing jobs they are programmed to do, could unintentionally make decisions that go against the values of its human users, such as physically harming people.
- **[Autonomous weapons] Ban the use of lethal autonomous weapons (LAWs)**: Lethal autonomous weapons (LAWs) are military robots that can attack targets without control by humans. LAWs could reduce the use of human combatants on the battlefield. But some worry that the adoption of LAWs could lead to mass violence. Because they are cheap and easy to produce in bulk, national militaries, terrorists, and other groups could readily deploy LAWs. 
- **[Technological unemployment] Guarantee a good standard of living for those who lose their jobs to automation**: Some forecast that AI will increasingly be able to do jobs done by humans today. AI could potentially do the jobs of blue-collar workers, like truckers and factory workers, as well as the jobs of white-collar workers, like financial analysts or lawyers. Some worry that in the future, robots and computers can do most of the jobs that are done by humans today. 
- **[Critical AI systems failure] Prevent critical AI systems failures**: As AI systems become more advanced, they could be used by the military or in critical infrastructure, like power grids, highways, or hospital networks. Some worry that the failure of AI systems or unintentional accidents in these applications could cause 10 percent or more of all humans to die. 

QUESTION: 

**In the next 10 years, how likely do you think it is that this AI governance challenge will impact large numbers of people in the U.S.?**

ANSWER CHOICES:

- Very unlikely: less than 5% chance (2.5%)
- Unlikely: 5-20% chance (12.5%)
- Somewhat unlikely: 20-40% chance (30%)
- Equally likely as unlikely: 40-60% chance (50%)
- Somewhat likely: 60-80% chance (70%)
- Likely: 80-95% chance (87.5%)
- Very likely: more than 95% chance (97.5%)
- I don’t know

QUESTION: 

**In the next 10 years, how likely do you think it is that this AI governance challenge will impact large numbers of people around the world?**

ANSWER CHOICES:

- Very unlikely: less than 5% chance (2.5%)
- Unlikely: 5-20% chance (12.5%)
- Somewhat unlikely: 20-40% chance (30%)
- Equally likely as unlikely: 40-60% chance (50%)
- Somewhat likely: 60-80% chance (70%)
- Likely: 80-95% chance (87.5%)
- Very likely: more than 95% chance (97.5%)
- I don’t know

QUESTION: 

**In the next 10 years, how important is it for tech companies and governments to carefully manage the following challenge?**

ANSWER CHOICES:

- Very important (3)
- Somewhat important (2)
- Not too important (1)
- Not at all important (0)
- I don't know

```{r govchallengetable, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
# Helper functions to generate the tables
# Likelihood in the US
gc_us_func <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = ai_gov[variable_number], 
              outcome_var = d[,paste0("Q8_", variable_number)], 
              shown = d0[,paste0("Q8_challenge_", variable_number)] == 1,
              label_var = d0[,paste0("Q8_", variable_number)], 
              num_missing = 98, num_DK = 8,
              new_values = c(mc_p_med, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d0[,paste0("Q8_challenge_", variable_number)] == 1))
  
}
# Likelihood around the world
gc_w_func <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = ai_gov[variable_number], 
              outcome_var = d[,paste0("Q9_", variable_number)], 
              shown = d0[,paste0("Q8_challenge_", variable_number)] == 1,
              label_var = d0[,paste0("Q9_", variable_number)], 
              num_missing = 98, num_DK = 8,
              new_values = c(mc_p_med, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d0[,paste0("Q8_challenge_", variable_number)] == 1))
  
}
# Issue importance 
gc_imp_func <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = ai_gov[variable_number], 
              outcome_var = d[,paste0("Q10_", variable_number)], 
              shown = d0[,paste0("Q8_challenge_", variable_number)] == 1,
              label_var = d0[,paste0("Q10_", variable_number)], 
              num_missing = 8, num_DK = 5,
              new_values = c(3:0, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d0[,paste0("Q8_challenge_", variable_number)] == 1))
  
}

# Frequency tables
likelihood_us <- gen_topline(group_var = ai_gov, varname = "Likelihood in the U.S.", 
            agg_func = gc_us_func)

likelihood_world <- gen_topline(group_var = ai_gov, varname = "Likelihood around the world", 
            agg_func = gc_w_func)

issue_importance <- gen_topline(group_var = ai_gov, varname = "Issue importance", 
            agg_func = gc_imp_func)

```

```{r govchallengetablelus1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[1]]
```

```{r govchallengetablelus2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[2]]
```

```{r govchallengetablelus3, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[3]]
```

```{r govchallengetablelus4, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[4]]
```

```{r govchallengetablelus5, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[5]]
```

```{r govchallengetablelus6, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[6]]
```

```{r govchallengetablelus7, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[7]]
```

```{r govchallengetablelus8, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[8]]
```

```{r govchallengetablelus9, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[9]]
```

```{r govchallengetablelus10, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[10]]
```

```{r govchallengetablelus11, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[11]]
```

```{r govchallengetablelus12, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[12]]
```

```{r govchallengetablelus13, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_us[[13]]
```

```{r govchallengetablelworld1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[1]]
```

```{r govchallengetablelworld2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[2]]
```

```{r govchallengetablelworld3, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[3]]
```

```{r govchallengetablelworld4, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[4]]
```

```{r govchallengetablelworld5, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[5]]
```

```{r govchallengetablelworld6, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[6]]
```

```{r govchallengetablelworld7, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[7]]
```

```{r govchallengetablelworld8, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[8]]
```

```{r govchallengetablelworld9, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[9]]
```

```{r govchallengetablelworld10, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[10]]
```

```{r govchallengetablelworld11, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[11]]
```

```{r govchallengetablelworld12, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[12]]
```

```{r govchallengetablelworld13, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
likelihood_world[[13]]
```

```{r govchallengetableissueimp1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[1]]
```

```{r govchallengetableissueimp2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[2]]
```

```{r govchallengetableissueimp3, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[3]]
```

```{r govchallengetableissueimp4, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[4]]
```

```{r govchallengetableissueimp5, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[5]]
```

```{r govchallengetableissueimp6, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[6]]
```

```{r govchallengetableissueimp7, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[7]]
```

```{r govchallengetableissueimp8, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[8]]
```

```{r govchallengetableissueimp9, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[9]]
```

```{r govchallengetableissueimp10, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[10]]
```

```{r govchallengetableissueimp11, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[11]]
```

```{r govchallengetableissueimp12, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[12]]
```

```{r govchallengetableissueimp13, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
issue_importance[[13]]
```

## Survey experiment: comparing perceptions of U.S. vs. China AI research and development {#airesearchcompare}

[Respondents were presented with one randomly-selected question from the two below.]

QUESTIONS:

- Compared with other industrialized countries, how would you rate the U.S. in AI research and development?
- Compared with other industrialized countries, how would you rate China in AI research and development?

ANSWER CHOICES:

- Best in the world (3)
- Above average (2)
- Average (1)
- Below average (0)
- I don’t know

```{r uschinard, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}

rd_countries <- c("U.S.", "China")

d0$Q12_1 <- d0$Q12a
d0$Q12_2 <- d0$Q12b
d$Q12_1 <- d$Q12a
d$Q12_2 <- d$Q12b

ai_rd_func <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = 
                           rd_countries[variable_number], 
              outcome_var = d[,paste0("Q12_", variable_number)], 
              shown = d0[,paste0("Q12_", variable_number)] != 9,
              label_var = d0[,paste0("Q12_", variable_number)], 
              num_missing = 8, num_DK = 5,
              new_values = c(1:4, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d0[,paste0("Q12_", variable_number)] != 9))
  
}

# Frequency tables
rd_results <- gen_topline(group_var = rd_countries, 
            varname = "Perceptions of research and development", 
            agg_func = ai_rd_func)

```

```{r uschinard1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
rd_results[[1]]
```

```{r uschinard2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
rd_results[[2]]
```

## Survey experiment: U.S.-China arms race {#armsraceexp}

[All respondents were presented with the following prompt.]

We want to understand your thoughts on some important issues in the news today. Please read the short news article below. 

Leading analysts believe that an "AI arms race" is beginning, in which the U.S. and China are investing billions of dollars to develop powerful AI systems for surveillance, autonomous weapons, cyber operations, propaganda, and command and control systems.

[Respondents were randomly assigned to one of the four experimental groups listed below.]

### Control

[No additional text.]

### Nationalism treatment

Some leaders in the U.S. military and tech industry argue that the U.S. government should invest much more resources in AI research to ensure that the U.S.'s AI capabilities stay ahead of China's. Furthermore, they argue that the U.S. government should partner with American tech companies to develop advanced AI systems, particularly for military use.

According to a leaked memo produced by a senior National Security Council official, China has "assembled the basic components required for winning the Al arms race...Much like America's success in the competition for nuclear weapons, China's 21st Century Manhattan Project sets them on a path to getting there first."

### War risks treatment

Some prominent thinkers are concerned that a U.S.-China arms race could lead to extreme dangers. To stay ahead, the U.S. and China may race to deploy advanced military AI systems that they do not fully understand or can control. We could see catastrophic accidents, such as a rapid, automated escalation involving cyber and nuclear weapons.

"Competition for AI superiority at [the] national level [is the] most likely cause of World War Three," warned Elon Musk, the CEO of Tesla and SpaceX.

### Common humanity treatment

Some prominent thinkers are concerned that a U.S.-China arms race could lead to extreme dangers. To stay ahead, the U.S. and China may race to deploy advanced military AI systems that they do not fully understand or can control. We could see catastrophic accidents, such as a rapid, automated escalation involving cyber and nuclear weapons.

"Unless we learn how to prepare for, and avoid, the potential risks, AI could be the worst event in the history of our civilization. It brings dangers, like powerful autonomous weapons," warned the late Stephen Hawking, one of the world's most prominent physicists. At the same time, he said that with proper management of the technology, researchers "can create AI for the good of the world."

[The order of the next two questions is randomized.]

QUESTION:

**How much do you agree or disagree with the following statement?**

>The U.S. should invest more in AI military capabilities to make sure it doesn't fall behind China's, even if doing so may exacerbate the arms race. For instance, the U.S. could increase AI research funding for the military and universities. It could also collaborate with American tech companies to develop AI for military use. 

ANSWER CHOICES: 

- Strongly agree (2)
- Somewhat agree (1)
- Neither agree nor disagree (0)
- Somewhat disagree (-1)
- Strongly disagree (-2)
- I don’t know

QUESTION:

**How much do you agree or disagree with the following statement?**

>The U.S. should work hard to cooperate with China to avoid the dangers of an AI arms race, even if doing so requires giving up some of the U.S.'s advantages. Cooperation could include collaborations between American and Chinese AI research labs, or the U.S. and China creating and committing to common safety standards.

ANSWER CHOICES:

- Strongly agree (2)
- Somewhat agree (1)
- Neither agree nor disagree (0)
- Somewhat disagree (-1)
- Strongly disagree (-2)
- I don’t know

```{r chinaexptable, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
# Helper functions to generate the tables

china_exp_1_func <- function(exp_group, output_type, surveyweights) {
  data.frame(catvar_func(outcome = ar_groups[exp_group], 
              outcome_var = d[,"Q12"], 
              shown = d$q12_treat_clean == ar_groups[exp_group],
              label_var = d0[,"Q12"], 
              num_missing = 8, num_DK = 6,
              new_values = c(2:-2, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d$q12_treat_clean == ar_groups[exp_group]))
  
}

china_exp_2_func <- function(exp_group, output_type, surveyweights) {
  data.frame(catvar_func(outcome = ar_groups[exp_group], 
              outcome_var = d[,"Q13"], 
              shown = d$q12_treat_clean == ar_groups[exp_group],
              label_var = d0[,"Q13"], 
              num_missing = 8, num_DK = 6,
              new_values = c(2:-2, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d$q12_treat_clean == ar_groups[exp_group]))
  
}

# Frequency tables
ar_results_invest <- gen_topline(group_var = ar_groups, 
            varname = "Responses to statement that U.S. should invest more in AI military capabilities", 
            agg_func = china_exp_1_func)

ar_results_coop <- gen_topline(group_var = ar_groups, 
            varname = "Responses to statement that U.S. should work hard to cooperate with China to avoid dangers of AI arms race", 
            agg_func = china_exp_2_func)
```

```{r chinaexptable1i, echo=FALSE, warning=FALSE, cache=TRUE}
ar_results_invest[[1]]
```

```{r chinaexptable2i, echo=FALSE, warning=FALSE, cache=TRUE}
ar_results_invest[[2]]
```

```{r chinaexptable3i, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
ar_results_invest[[3]]
```

```{r chinaexptable4i, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
ar_results_invest[[4]]
```

```{r chinaexptable1c, echo=FALSE, warning=FALSE, cache=TRUE}
ar_results_coop[[1]]
```

```{r chinaexptable2c, echo=FALSE, warning=FALSE, cache=TRUE}
ar_results_coop[[2]]
```

```{r chinaexptable3c, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
ar_results_coop[[3]]
```

```{r chinaexptable4c, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
ar_results_coop[[4]]
```

## Issue areas for possible U.S.-China cooperation {#uschinacoop}

QUESTION: 

**For the following issues, how likely is it that the U.S. and China can cooperate?**

[Respondents were presented with three issues from the list below. All three issues were presented on the same page; the order that they appeared was randomized.]

- Prevent AI cyber attacks against governments, companies, organizations, and individuals.
- Prevent AI-assisted surveillance from violating privacy and civil liberties.
- Make sure AI systems are safe, trustworthy, and aligned with human values.
- Ban the use of lethal autonomous weapons.
- Guarantee a good standard of living for those who lose their jobs to automation.

ANSWER CHOICES:

- Very unlikely: less than 5% chance (2.5%)
- Unlikely: 5-20% chance (12.5%)
- Somewhat unlikely: 20-40% chance (30%)
- Equally likely as unlikely: 40-60% chance (50%)
- Somewhat likely: 60-80% chance (70%)
- Likely: 80-95% chance (87.5%)
- Very likely: more than 95% chance (97.5%)
- I don’t know

```{r coopissue, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
# Helper functions to generate the tables

coop_issue <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = coop_outcome[variable_number], 
              outcome_var = d[,paste0("Q14_", variable_number)], 
              shown = d0[,paste0("Q14_", variable_number)] != 99,
              label_var = d0[,paste0("Q14_", variable_number)], 
              num_missing = 98, num_DK = 8,
              new_values = c(mc_p_med, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d0[,paste0("Q14_", variable_number)] != 99))
  
}

coop_outcome_results <- gen_topline(group_var = coop_outcome, 
            varname = "Likelihood of cooperation with China", 
            agg_func = coop_issue)

```

```{r coopoutcome1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
coop_outcome_results[[1]]
```

```{r coopoutcome2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
coop_outcome_results[[2]]
```

```{r coopoutcome3, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
coop_outcome_results[[3]]
```

```{r coopoutcome4, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
coop_outcome_results[[4]]
```

```{r coopoutcome5, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
coop_outcome_results[[5]]
```

## Trend across time: job creation or job loss {#jobtime}

QUESTION:

**How much do you agree or disagree with the following statement?**

[Respondents were presented with one statement randomly selected from the list below.]

- In general, automation and AI will create more jobs than they will eliminate.
- In general, automation and AI will create more jobs than they will eliminate in 10 years.
- In general, automation and AI will create more jobs than they will eliminate in 20 years.
- In general, automation and AI will create more jobs than they will eliminate in 50 years.

ANSWER CHOICES:

- Strongly agree (2)
- Agree (1)
- Disagree (-1)
- Strongly disagree (-2)
- I don’t know


```{r timeframe, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
# Helper functions to generate the tables

time_frame_func <- function(exp_group, output_type, surveyweights) {
  data.frame(catvar_func(outcome = time_exp[exp_group], 
              outcome_var = d[,"Q15"], 
              shown = d$q15_treat == exp_group,
              label_var = d0[,"Q15"], 
              num_missing = 8, num_DK = 5,
              new_values = c(2, 1, -1, 2, NA, NA, NA),
              survey_weights = surveyweights, edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = sum(d$q15_treat == exp_group))
  
}

# Frequency tables
time_exp_results <- gen_topline(group_var = time_exp, 
            varname = "Responses to statement that automation and AI will create more jobs than they will eliminate", 
            agg_func = time_frame_func)

```

```{r timeexpout1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
time_exp_results[[1]]
```

```{r timeexpout2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
time_exp_results[[2]]
```

```{r timeexpout3, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
time_exp_results[[3]]
```

```{r timeexpout4, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
time_exp_results[[4]]
```

## High-level machine intelligence: forecasting timeline {#forecasthlmi}

QUESTION:

The following questions ask about high-level machine intelligence. We have high-level machine intelligence when machines are able to perform almost all tasks that are economically relevant today better than the median human (today) at each task. These tasks include asking subtle common-sense questions such as those that travel agents would ask. For the following questions, you should ignore tasks that are legally or culturally restricted to humans, such as serving on a jury. 

**In your opinion, how likely is it that high-level machine intelligence will exist in 10 years? 20 years? 50 years? For each prediction, please use the slider to indicate the percent chance that you think high-level machine intelligence will exist. 0% chance means it will certainly not exist. 100% chance means it will certainly exist.**

______ In 10 years?

______ In 20 years?

______ In 50 years?

ANSWER CHOICES:

- Very unlikely: less than 5% chance (2.5%)
- Unlikely: 5-20% chance (12.5%)
- Somewhat unlikely: 20-40% chance (30%)
- Equally likely as unlikely: 40-60% chance (50%)
- Somewhat likely: 60-80% chance (70%)
- Likely: 80-95% chance (87.5%)
- Very likely: more than 95% chance (97.5%)
- I don’t know

```{r hlmitimelinetable, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, results='asis'}
# Helper functions to generate the tables
future_years <- c("10 years", "20 years", "50 years")
hlmi_timeline_func <- function(variable_number, output_type, surveyweights) {
  data.frame(catvar_func(outcome = future_years[variable_number], 
              outcome_var = d[,paste0("Q16_", variable_number)], 
              shown = rep(TRUE, nrow(d)),
              label_var = d0[,paste0("Q16_", variable_number)], 
              num_missing = 98, num_DK = 8,
              new_values = c(mc_p_med, NA, NA, NA),
              survey_weights = surveyweights, 
              edit_labels = FALSE,
              missing_recode = 0,
              output_type), 
             n_show = 2000)
  
}

future_years_results <- gen_topline(group_var = future_years, 
            varname = "Forecasting high-level machine intelligence", 
            agg_func = hlmi_timeline_func)

```

```{r futureyear1, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
future_years_results[[1]]
```

```{r futureyear2, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
future_years_results[[2]]
```

```{r futureyear3, echo=FALSE, warning=FALSE, cache=TRUE, results='asis'}
future_years_results[[3]]
```

## Support for developing high-level machine intelligence {#supporthlmi}

QUESTION: 

**How much do you support or oppose the development of high-level machine intelligence?** 

ANSWER CHOICES:

- Strongly support
- Somewhat support
- Neither support nor oppose
- Somewhat oppose
- Strongly oppose
- I don’t know

```{r hlmisupportdevtable, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE}

supportdev_h_w <- catvar_func(
  outcome = label(d0$Q17),
  outcome_var = d$Q17,
  label_var = d0$Q17,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(2, 1, 0, -1, -2, NA, NA, NA),
  survey_weights = d$survey_weights,
  missing_recode = 0
  )

supportdev_h_u <- catvar_func(
  outcome = label(d0$Q17),
  outcome_var = d$Q17,
  label_var = d0$Q17,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(2, 1, 0, -1, -2, NA, NA, NA),
  survey_weights = d$weight1,
  missing_recode = 0
  )

supportdev_h_d <- data.frame(labels = supportdev_h_w$labels, 
                           w_perc = roundfunc(100*supportdev_h_w$Prop, 2),
                           u_perc = roundfunc(100*supportdev_h_u$Prop, 2),
                           freq = supportdev_h_u$Freq)

kable(x = supportdev_h_d, 
      format = "pandoc",
                   caption = paste0("Support for developing high-level machine intelligence; \\textit{N} = 2000"),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies")) 


```

## Expected outcome of high-level machine intelligence {#expectedoutcome}

QUESTION: 

Suppose that high-level machine intelligence could be developed one day. How positive or negative do you expect the overall impact of high-level machine intelligence to be on humanity in the long run? 

ANSWER CHOICES:

- Extremely good 
- On balance good
- More or less neutral
- On balance bad
- Extremely bad, possibly human extinction
- I don't know

```{r hlmisupportexptable, echo=FALSE, warning=FALSE, cache=TRUE}

exph_w <- catvar_func(
  outcome = label(d0$Q18),
  outcome_var = d$Q18,
  label_var = d0$Q18,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(2:-2, NA, NA, NA),
  survey_weights = d$survey_weights,
  missing_recode = 0
  )

exph_u <- catvar_func(
  outcome = label(d0$Q17),
  outcome_var = d$Q18,
  label_var = d0$Q18,
  output_type = "value_table",
  shown = rep(TRUE, nrow(d)),
  num_missing = 8,
  num_DK = 6, edit_labels = FALSE,
  new_values <- c(2:-2, NA, NA, NA),
  survey_weights = d$weight1,
  missing_recode = 0
  )

exph_d <- data.frame(labels = exph_w$labels, 
                           w_perc = roundfunc(100*exph_w$Prop, 2),
                           u_perc = roundfunc(100*exph_u$Prop, 2),
                           freq = exph_u$Freq)

kable(x = exph_d, 
      format = "pandoc",
                   caption = paste0("Expected outcome of high-level machine intelligence; \\textit{N} = 2000"),
        col.names = c("Answer choices", 
                      "Percentages (weighted)", "Percentages (unweighted)",
                      "Raw frequencies")) 


```

\newpage

# Appendix C: Additional data analysis results {#addresults}

## Support for developing AI {#addsupportdevai}

Table \@ref(tab:demosupportaitable) shows the regression results used to produce Figure \@ref(fig:demographicsupport2).

```{r demosupportaitable, echo=FALSE, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}

kable(x = dev_md_support_ai_k, row.names = FALSE, col.names = c("Variables", "Coefficients (SEs)"),
      format = "pandoc", caption = "Predicting support for developing AI using demographic characteristics: results from a multiple linear regression that includes all demographic variables; outcome standardized to have mean 0 and unit variance")

```

## Survey experiment and cross-national comparison: AI and/or robots should be carefully managed {#addcarefullym}

We present the percentage of "don't know" or missing responses to the survey question (see [Appendix B](#manageexp) for the survey question text). Regression analysis shows that the varying the term used (i.e., AI, AI and robots, and robots) does not change responses to the statement that such technologies should be carefully managed. This finding is robust to a regression where we controlled for "don't know" or missing responses. In Table \@ref(tab:aicomparetable), we present the distribution of responses to the statement by country.

```{r aimanagedexpadd1, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}

# Attrition check
# Attrition rates by experimental groups
q5b_attrition <- d %>% group_by(q5b_treat_clean) %>% dplyr::summarise(
  attrition_prop = mean(Q5b_missing)*100,
  DK_prop = mean(Q5b == 5)*100
) 
q5b_attrition$missing_prop <- q5b_attrition$attrition_prop - q5b_attrition$DK_prop
kable(x = q5b_attrition, format = "pandoc",
      caption = "Survey experiment attrition check: agreement with statement that AI and/or robots should be carefully managed", col.names = c("Experimental condition", "Percent DK/missing", "Percent DK", "Percent missing"), digits = 2)
```

```{r aimanagedexpadd2, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
# Attrition check: regression results 
if (sum(q5b_attrition$attrition_prop > 0) > 0) {
  regression_output(formula = "Q5b_missing ~ q5b_treat_clean", shown = rep(TRUE, nrow(d)),
                    survey_weights = d$survey_weights,
                  variable_names = c("(Intercept)", "AI and robots", "Robots"),
                  caption = "Survey experiment attrition check: agreement with statement that AI and/or robots should be carefully managed")  
}
```

```{r aimanagedexpadd3a, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
d$q5b_treat_clean_ai_robots <- d$q5b_treat_clean == "AI"
  d$q5b_treat_clean_robots <- d$q5b_treat_clean == "Robots"
  
regression_output(formula = "Q5b_clean ~ q5b_treat_clean", shown = rep(TRUE, nrow(d)),
                    survey_weights = d$survey_weights,
                  variable_names = c("(Intercept)", "AI and robots", "Robots"),
                  caption = "Survey experiment results: agreement with statement that AI and/or robots should be carefully managed")

```

```{r aimanagedexpadd3b, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
regression_output(formula = "Q5b_clean ~ q5b_treat_clean_ai_robots + q5b_treat_clean_robots + scale(Q5b_missing) + q5b_treat_clean_ai_robots:scale(Q5b_missing) + q5b_treat_clean_robots:scale(Q5b_missing)", shown = rep(TRUE, nrow(d)),
                    survey_weights = d$survey_weights,
                  variable_names = c("(Intercept)", "AI and robots", "Robots",
                                     "DK/missing", "AI and robots x DK/missing",
                                     "Robots x DK/missing"),
                  caption = "Survey experiment results: agreement with statement that AI and/or robots should be carefully managed (controlling for DK/missing responses)",
                  kable_output = c(1:3, 7))  
```

```{r aicomparetable, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
kable(euro_com_w, format = "pandoc",  
      col.names = c("Countries", "Totally disagree", "Tend to disagree", "Tend to agree",
                    "Totally agree", "Don't know"), row.names = FALSE,
      caption = "Distribution of responses to statement that AI and robots should be carefully managed by country (in percentages); EU countries data from Eurobarometer")

```

## Harmful consequences of AI in the context of other global risks {#appglobalrisks}

Table \@ref(tab:appglobalrisks) summarizes responses to 15 potential global risks. 

```{r appglobalrisks, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
# Make the table
gr_sum_table <- gr_sum
gr_sum_table$prob <- paste0(round(gr_sum_table$prob), "%")
gr_sum_table$impact <- roundfunc(gr_sum$impact)
# Outcome: number of selected items
kable(gr_sum_table, row.names = FALSE, format = "pandoc",
      col.names = c("Potential risks",
                                                     "Mean perceived likelihood",
                                                     "Mean perceived impact", "\\textit{N}"),
      caption = "Summary statistics: the American public’s perceptions of 15 potential global risks")
```

## Survey experiment: what the public considers AI, automation, machine learning, and robotics {#aawhatsai}

We formally tested whether or not respondents think AI, automation, machine learning, and robotics are used in different applications. (See [Appendix B](#considersai) for the survey question text.) For each technological application, we used an $F$-test to test whether any of terms randomly assigned to the respondents affect respondents' selecting that application. Because we ran 10 $F$-tests, we used the Bonferroni correction to control the familywise error rate. The Bonferroni correction rejected the null hypothesis at alpha level $\alpha/10$, instead of $\alpha$. For instance, to test whether the $F$-static is significant at the 5% level, we set the alpha level at $\alpha/10 = 0.005$. Our results (in Table \@ref(tab:Ftestwhat)) show that except for social robots, respondents think that AI, automation, machine learning, and robotics are used in each of the applications presented in the survey. 

```{r Ftestwhat, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}

# F-tests
def_analysis <- function(varname) {
  reg_sum <- summary(lm(as.formula(paste0(varname, " == 1 ~ q3new_treat_c")), 
                      data = d, 
                     weights = d$survey_weights))
f_stat_p <- pf(reg_sum$fstatistic[1],
                 reg_sum$fstatistic[2],
                 reg_sum$fstatistic[3],
                 lower.tail = FALSE) %>% as.numeric()
f_stat <- c(reg_sum$fstatistic, p_value = f_stat_p)
return(f_stat)
}
# Make the table
f_stat_res <- data.frame(applications = rev(levels(whatsai_d$variable)) ,
                         do.call(rbind, lapply(X = paste0("Q3new_", 1:10),
                                               def_analysis)))
# Clean up the table
f_stat_res$f_stat <-
    paste0(
      "\\textit{F}(",
      f_stat_res$numdf,
      ", ",
      f_stat_res$dendf,
      ") = ",
      roundfunc(f_stat_res$value)
    )
f_stat_res$p_value_c <- ifelse(f_stat_res$p_value < 0.001, "<0.001", round(f_stat_res$p_value, 3))
f_stat_res$sig <- ifelse(f_stat_res$p_value < 0.05/10, "Yes", "No")

# Make the kable table
kable(f_stat_res[,c("applications", "f_stat", "p_value_c", 
                    "sig")], format = "pandoc",  
      col.names = c("Technological applications", "\\textit{F}-statistic", "\\textit{p}-value",
                    "Significant"), 
      caption = "Respondents distinguish between AI, automation, machine learning, and robotics")

```

Next, we investigated the problem of respondents not selecting technological applications where it would be logical to pick them (e.g., not selecting industrial robots or social robots when presented with the term "robotics"). Our regression analysis shows that this type of non-response is correlated with respondents' inattention. 
 
We used two measures as a proxy for inattention: 

1. time to complete the survey
2. the absolute deviation from the median time to complete the survey. 

Because the distribution of completion times is heavily skewed right, we used absolute deviation from the median, as opposed to the mean. The median is 13 minutes whereas the mean is 105 minutes. We incorporated the second measure because we suspected that people who took very little time _or_ a very long time to complete the survey were inattentive. 

We used three outcomes that measured non-response:

1. the number of items selected
2. not selecting "none of the above"
3. selecting items containing the word "robots" for respondents assigned to consider "robotics"

Using multiple regression, we showed that inattention predicts non-response measured by the three outcomes above (see Tables \@ref(tab:termsattentioncheck1), \@ref(tab:termsattentioncheck2), and \@ref(tab:termsattentioncheck3)). 

```{r termsattentioncheck1, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}

# Outcome: number of selected items
regression_output(shown = rep(TRUE, nrow(d)), survey_weights = d$weight,
                  formula = "whatsai_s ~ surveytime + surveytime_dev + q3new_treat_c", variable_names = c("(Intercept)",
                    "Survey completion time (min)", 
  "Absolute deviation from median survey completion time (min)", "Term: automation", "Term: machine learning", "Term: Robotics"), 
                  caption = "Correlation between survey completion time and number of selected items", kable_output = TRUE)
```

```{r termsattentioncheck2, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
# Outcome: selecting none of the above
regression_output(formula = "as.numeric(d$Q3new_11 == 2) ~ surveytime + surveytime_dev + q3new_treat_c", variable_names = c("(Intercept)",
                    "Survey completion time (min)", 
  "Absolute deviation from median survey completion time (min)", "Term: automation", "Term: machine learning", "Term: Robotics"), 
                  caption = "Correlation between survey completion time and not selecting 'none of the above'", kable_output = TRUE, survey_weights = d$weight)
```

```{r termsattentioncheck3, echo=FALSE, fig.height=3, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, warning=FALSE}
# Not selecting robots
regression_output(shown = d$q3new_treat == 4, survey_weights = d$weight[d$q3new_treat == 4],
                  formula = "whatsrobot_s ~ surveytime + surveytime_dev", variable_names = c("(Intercept)",
                    "Survey completion time (min)", 
  "Absolute deviation from median survey completion time (min)"), 
                  caption = "Correlation between survey completion time and selecting 'robots' when assigned the term 'robotics'", kable_output = TRUE)

```

## AI governance challenges: prioritizing governance challenges {#appgovchallenges}

We compared respondents' perceived likelihood of each governance challenge impacting large numbers of people in the U.S. with respondents' perceived likelihood of each governance challenge impacting large numbers of people around the world. (See [Appendix B](#govchallenges) for the survey question text.) For each governance challenge, we used linear regression to estimate the difference between responses to the U.S. question and the world question.

Because we ran 13 tests, we used the Bonferroni correction to control the familywise error rate. In our case, the Bonferroni correction rejected the null hypothesis at alpha level $\alpha/13$, instead of $\alpha$. To test whether the differences are significant at the 5% level, we set the alpha level at $\alpha/13 = 0.004$. According to Table \@ref(tab:airisks3), Americans perceive that all governance challenges, except for protecting data privacy and ensuring safe autonomous vehicles, are more likely to impact people around the world than in the U.S. specifically. In particular, Americans think that autonomous weapons are 7.6 percentage points more likely to impact people around the world than in the U.S.

```{r airisks3, echo=FALSE, fig.height=7, fig.keep='all', cache=TRUE, warning=FALSE, warning=FALSE}

# Clean up the data

ai_gov_uw_diff_c <- ai_gov_uw_diff
ai_gov_uw_diff_c$mean_us <- 
  roundfunc(ai_gov_uw_diff_c$mean_us, 1)
ai_gov_uw_diff_c$num <- roundfunc(ai_gov_uw_diff_c$num, 1)
ai_gov_uw_diff_c$se <- roundfunc(ai_gov_uw_diff_c$se, 1)
ai_gov_uw_diff_c$num_se <- paste0(ai_gov_uw_diff_c$num,
                                  " (", ai_gov_uw_diff_c$se,
                                  ")")
ai_gov_uw_diff_c$pvalue_c <- ifelse(ai_gov_uw_diff_c$pvalue < 0.001, "<0.001", roundfunc(ai_gov_uw_diff_c$pvalue, 3))
ai_gov_uw_diff_c$sig <-
  ifelse(ai_gov_uw_diff_c$pvalue < 0.05/nrow(ai_gov_uw_diff),
         "Yes", "No")

# Make the kable table

kable(ai_gov_uw_diff_c[,c("gov_challenge", "mean_us", "num_se",
                    "pvalue_c", "sig")], format = "pandoc",  
      col.names = c("Governance challenge", "U.S. mean likelihood", "Difference (SE)", "\\textit{p}-value",
                    "Significant"), 
      caption = "Comparing perceived likelihood: in U.S. vs. around the world; each difference is the U.S. mean likelihood subtracted from the world mean likelihood")

```

To highlight the differences between the responses of demographic subgroups regarding issue importance, we created an additional graph (Figure \@ref(fig:airisksdemo2)). Here, we subtracted the overall mean of perceived issue importance across all responses from each subgroup-governance challenge mean. [^diffmean] Table \@ref(tab:aigovregsat) shows the results from a saturated regression predicting perceived issue importance using demographic variables, AI governance challenge, and interactions between the two types of variables.  

[^diffmean]: Note that the perceived issue importance was measured on a four-point scale, where 0 meant "not at all important" and 3 meant "very important." We only mean-centered the outcomes; we did not standardize such that the outcomes have unit variance. 

```{r aigovchallengetableus, echo=FALSE, fig.height=10, fig.keep='all', fig.width=7, fig.cap = "table governance challenges", cache=TRUE, warning=FALSE, warning=FALSE}

# US Table
ag_sum_US$product <- ag_sum_US$prob/100 * ag_sum_US$importance
ag_sum_US$prob_c <- paste0(roundfunc(ag_sum_US$prob, round_digits = 0), "%")
ag_sum_US$importance_c <- paste0(roundfunc(ag_sum_US$importance))
ag_sum_US <- ag_sum_US[order(ag_sum_US$product, decreasing = TRUE),]
ag_sum_US$product <- roundfunc(ag_sum_US$product)

kable(ag_sum_US[,c("gov_challenge", "prob_c", "importance_c", "product")], format = "pandoc",  
      col.names = c("Governance challenge", "Mean likelihood", "Mean issue importance", 
                    "Product of likelihood and issue importance"), row.names = FALSE,
      caption = "Perception of AI governance challenges in the U.S.: summary statistics table")
```

```{r aigovchallengetableworld, echo=FALSE, fig.height=10, fig.keep='all', fig.width=7, fig.cap = "table governance challenges", cache=TRUE, warning=FALSE, warning=FALSE}
# World Table
ag_sum_world$product <- ag_sum_world$prob/100 * ag_sum_world$importance
ag_sum_world$prob_c <- paste0(roundfunc(ag_sum_world$prob, round_digits = 0), "%")
ag_sum_world$importance_c <- paste0(roundfunc(ag_sum_world$importance))
ag_sum_world <- ag_sum_world[order(ag_sum_world$product, decreasing = TRUE),]
ag_sum_world$product <- roundfunc(ag_sum_world$product)

kable(ag_sum_world[,c("gov_challenge", "prob_c", "importance_c", "product")], format = "pandoc",  
      col.names = c("Governance challenge", "Mean likelihood", "Mean issue importance", 
                    "Product of likelihood and issue importance"), row.names = FALSE,
      caption = "Perception of AI governance challenges in the world: summary statistics table")

```

```{r airisksdemo2, echo=FALSE, fig.height=10, fig.keep='all', fig.width=7, fig.cap = "AI governance challenges: issue importance by demographic subgroups", cache=TRUE, warning=FALSE, warning=FALSE}
# Make second graph
ggplot(heat_map_res, aes(x = gov_challenge, y = characteristic, fill = importance_mc))+
  geom_bin2d(color = "white") + xlab("AI governance challenges") + 
  scale_y_discrete(name = "Demographic subgroups",
                   labels = function(x) str_wrap(x, width = 30)) +
  # geom_text(aes(x = gov_challenge, y = characteristic, 
  #               label = roundfunc(importance_mc, 1)), color = "black", size = 3) + 
  scale_fill_gradient2(name = "Mean-centered issue importance on a 4-point scale\n(Smaller value = less important; Greater value = more important)", 
                       low = "#1b7837", mid = "#f7f7f7", high = "#762a83") + 
    theme_bw() + theme(legend.position = "bottom", 
                       axis.text.x = element_text(angle = 270, hjust = 0)) +
  guides(fill = guide_colourbar(title.position="top", title.hjust = 0.5, 
                                barwidth = 15)) +
  labs(caption = "Source: Center for the Governance of AI")
```


```{r aigovregsat, echo=FALSE, fig.height=10, fig.keep='all', fig.width=7, fig.cap = "table governance challenges", cache=TRUE, warning=FALSE, warning=FALSE}

kable(ai_imp_regc, format = "pandoc",  
      col.names = c("Variables", 
                    "Coefficient (SEs)"), 
      row.names = FALSE,
      caption = "Results from a saturated regression predicting perceived issue importance using demographic variables, AI governance challenge, and interactions between the two types of variables; the coefficients for the interactions variables are not shown due to space constraints")

```

## Trust in various actors to develop and manage AI in the interest of the public

Table \@ref(tab:trustapptableapp) displays the mean level of trust the public expresses in various actors to develop and manage AI in the interest of the public.

```{r trustapptableapp, echo=FALSE, cache=TRUE, warning=FALSE}

# Output the table via kable
kable(trust_table[,c("outcome", "dev_ai", "manage_ai")], 
      caption = "Trust in various actors to develop and manage AI in the interest of the public: mean responses", 
      row.names = FALSE, format = "latex", booktabs = TRUE,
      col.names = c("Actors",
                    "Trust to develop AI", "Trust to manage AI")) %>%
  column_spec(column = c(1:3), width = "4cm") 

```

\newpage

## Survey experiment: comparing perceptions of U.S. vs. China AI research and development {#appuschinacomp}

A substantial percentage of respondents selected "I don't know" when answering this survey question. (See [Appendix B](#airesearchcompare) for the survey question text.) Our regression analysis shows that there is a small but statistically significant difference between respondents' perception of R&D in the U.S. as compared to in China, as seen in Tables \@ref(tab:uschinaapp3a) and \@ref(tab:uschinaapp3b).

```{r uschinaapp1, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf'}
# Clean the data for regression analysis
d$Q12ab_clean <- relabel_var(ifelse(is.na(d$Q12a) | d$Q12a == 9, d$Q12b, d$Q12a),
                                        c(1:5, 8, 9), c(3, 2, 1, 0, NA, NA, NA))
d$Q12ab_missing <- is.na(d$Q12ab_clean)
d$Q12ab_clean[is.na(d$Q12ab_clean)] <- rd_overall_mean
d$q12a_treat_clean <- relabel_var(d$q12a_treat, c(1, 2, 8, 9), c("U.S.", "China", NA, NA))
# Attrition check
# Attrition rates by experimental groups
d$Q12_ab <- ifelse(is.na(d$Q12a) | d$Q12a == 9, d$Q12b, d$Q12a)
q12ab_attrition <- d %>% group_by(q12a_treat_clean) %>% dplyr::summarise(
  attrition_prop = mean(Q12ab_missing)*100,
  DK_prop = mean(Q12_ab == 5)*100
) 
q12ab_attrition$missing_prop <- q12ab_attrition$attrition_prop - q12ab_attrition$DK_prop
kable(x = q12ab_attrition, format = "pandoc",
      caption = "Survey experiment attrition check: comparing U.S. and China's AI research and development", col.names = c("Experimental condition", "Percent DK/missing", "Percent DK", "Percent missing"), digits = 2)
```

```{r uschinaapp2, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf'}
# Attrition check: regression results 
if (sum(q12ab_attrition$attrition_prop > 0) > 0) {
  regression_output(formula = "Q12ab_missing ~ q12a_treat_clean", 
                  variable_names = c("(Intercept)", "U.S."), shown = rep(TRUE, nrow(d)),
                    survey_weights = d$survey_weights,
                  caption = "Survey experiment attrition check: comparing U.S. and China's AI research and development") 
}
```


```{r uschinaapp3a, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf'}
regression_output(
    formula = "Q12ab_clean ~ q12a_treat_clean",
    variable_names = c("(Intercept)", "U.S."),
    shown = rep(TRUE, nrow(d)),
    survey_weights = d$survey_weights,
    caption = "Survey experiment results: comparing U.S. and China's AI research and development"
  )
```

```{r uschinaapp3b, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, cache=TRUE, warning=FALSE, dpi = 300, dev = 'pdf'}
regression_output(
  formula = "Q12ab_clean ~ q12a_treat_clean + scale(Q12ab_missing) + q12a_treat_clean:scale(Q12ab_missing)",
  shown = rep(TRUE, nrow(d)),
  survey_weights = d$survey_weights,
  variable_names = c("(Intercept)", "U.S.", "DK/missing", "U.S. x DK/missing"),
  caption = "Survey experiment results: comparing U.S. and China's AI research and development (controlling for DK/missing responses)",
  kable_output = c(1:2, 5))
  
```

\FloatBarrier

## Survey experiment: U.S.-China arms race {#appuschinaarmsrace}

We checked that "don't know" or missing responses to both statements are not induced by the information treatments. (See [Appendix B](#armsraceexp) for the survey experiment text.) Next, we examined the correlation between responses to the two statements using a 2D bin count graph. The overall Pearson correlation coefficient is -0.05 but there exists considerable variation by experimental condition. 

```{r armsracecorrmain1, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
kable(
  x = q12_attrition,
  format = "pandoc",
  caption = "Survey experiment attrition check: agreement with statement that U.S. should invest more in AI military capabilities",
  col.names = c(
    "Experimental condition",
    "Percent DK/missing",
    "Percent DK",
    "Percent missing"
  ),
  digits = 2
)

```
   
```{r armsracecorrmain2, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
# Attrition check: regression results
if (sum(q12_attrition$attrition_prop > 0) > 0) {
  regression_output(
    formula = "Q12_missing ~ q12_treat_clean",
    shown = rep(TRUE, nrow(d)),
    survey_weights = d$survey_weights,
    variable_names = c("(Intercept)", ar_groups[2:4]),
    caption = "Survey experiment attrition check: agreement with statement that U.S. should invest more in AI military capabilities"
  )
}

```
    
```{r armsracecorrmain3, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
kable(
  x = q13_attrition,
  format = "pandoc",
  caption = "Survey experiment attrition check: agreement with statement that U.S. should work hard to cooperate with China to avoid dangers of AI arms race",
  col.names = c(
    "Experimental condition",
    "Percent DK/missing",
    "Percent DK",
    "Percent missing"
  ),
  digits = 2
)
```

```{r armsracecorrmain4, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
# Attrition check: regression results
if (sum(q13_attrition$attrition_prop > 0) > 0) {
  regression_output(
    formula = "Q13_missing ~ q13_treat_clean",
    shown = rep(TRUE, nrow(d)),
    survey_weights = d$survey_weights,
    variable_names = c("(Intercept)", ar_groups[2:4]),
    caption = "Survey experiment attrition check: agreement with statement that U.S. should work hard to cooperate with China to avoid dangers of AI arms race"
  )
}
```

```{r armsracecorrfig, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, fig.cap = "Correlation between responses to the two statements from survey experiment", warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}

# Make a new variable
d$Q12_corr <- d$Q12_clean
d$Q12_corr[!d$Q12_corr %in% c(-2:2)] <- 3

d$Q13_corr <- d$Q13_clean
d$Q13_corr[!d$Q13_corr %in% c(-2:2)] <- 3

# Convert to factor
d$Q12_corr <- factor(
  d$Q12_corr,
  levels = c(-2:3),
  labels = c(
    "Strongly\ndisagree",
    "Somewhat\ndisagree",
    "Neither agree\nnor disagree",
    "Somewhat\nagree",
    "Strongly\nagree",
    "DK"
  )
)
d$Q13_corr <- factor(
  d$Q13_corr,
  levels = c(-2:3),
  labels = c(
    "Strongly\ndisagree",
    "Somewhat\ndisagree",
    "Neither agree\nnor disagree",
    "Somewhat\nagree",
    "Strongly\nagree",
    "DK"
  )
)

# Summarize the data
q13_12 <- d %>% group_by(Q12_corr, Q13_corr) %>%
  dplyr::summarise(prop = n() / nrow(d),
                   percent = roundfunc(n() / nrow(d) * 100, 1))

# Weighted correlation
survey_experiment_corr <-
  weightedCorr(
    x = d$Q12_clean,
    y = d$Q13_clean,
    weights = d$survey_weights,
    method = "pearson"
  )

# Make the graph
ggplot() +
  stat_summary_2d(data = q13_12, aes(x = Q12_corr,
                                     y = Q13_corr, z = prop)) +
  geom_shadowtext(
    data = q13_12,
    aes(x = Q12_corr,
        y = Q13_corr, label = percent),
    color = "black",
    bg.colour = "white",
    bg.r = 0.05
  ) +
  theme_bw() +
  geom_smooth(
    data = d,
    aes(x = Q12_clean + 3, y = Q13_clean + 3, weight = survey_weights),
    method = "lm",
    se = FALSE,
    linetype = 2,
    color = "red"
  ) +
  xlab(
    str_wrap(
      "Agreement with statement that U.S. should invest more in AI military capabilities",
      40
    )
  ) +
  ylab(
    str_wrap(
      "Agreement with statement that U.S. should work hard to cooperate with China to avoid dangers of AI arms race",
      40
    )
  ) + coord_equal() +
  labs(title = paste0("Pearson's r = ", roundfunc(survey_experiment_corr)),
       caption = "Source: Center for the Governance of AI")  +
  scale_fill_gradient(
    name = "Percentage of respondents",
    labels = scales::percent,
    breaks = c(0, 2, 4, 6, 8, 10, 12) / 100,
    limits = c(0, 12) / 100,
    low = "#f7fbff",
    high = "#08519c"
  ) +
  theme(legend.position = "bottom",
        axis.text.x = element_text(angle = 270, hjust = 0)) +
  guides(fill = guide_colourbar(
    title.position = "top",
    title.hjust = 0.5,
    barwidth = 12
  ))
```

```{r armsracecorrtab, echo=FALSE, fig.height=5, fig.keep='all', fig.width=7, fig.cap = "Correlation between responses to the two statements from survey experiment", warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
corr_table <-
  d %>% group_by(q13_treat_clean) %>% dplyr::summarise(
    wcorr = wCorr::weightedCorr(
      x = Q12_clean,
      y = Q13_clean,
      weights = survey_weights,
      method = "pearson"
    )
  )

corr_table <-
  rbind(data.frame(
    q13_treat_clean = "Overall",
    wcorr = wCorr::weightedCorr(
      x = d$Q12_clean,
      y = d$Q13_clean,
      weights = d$survey_weights,
      method = "pearson"
    )
  ),
  corr_table)

kable(
  x = corr_table,
  format = "pandoc",
  caption = "Correlation between responses to the two statements",
  col.names = c("Experimental condition",
                "Pearson correlation"),
  digits = 2
)
```

\FloatBarrier 

## Trend across time: job creation or job loss {#appjobloss}

There are many "don't know" responses to this survey question (see [Appendix B](#jobtime) for the survey question text). Nevertheless, "don't know" or missing responses are not affected by the experimental future time framing. $F$-tests reveal that there are no differences in responses to the three future time frames, as seen in Table \@ref(tab:attritiontimeline4).

```{r attritiontimeline1, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
# Attrition rates by experimental groups
q15_attrition <-
  d %>% group_by(q15_treat_clean) %>% dplyr::summarise(attrition_prop = mean(Q15_missing) *
                                                         100,
                                                       DK_prop = mean(Q15 == 5) * 100)
q15_attrition$missing_prop <-
  q15_attrition$attrition_prop - q15_attrition$DK_prop
kable(
  x = q15_attrition, format = "pandoc",
  caption = "Survey experiment attrition check: future time frame",
  col.names = c(
    "Experimental condition",
    "Percent DK/missing",
    "Percent DK",
    "Percent missing"
  ),
  digits = 2
)
```

```{r attritiontimeline2, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
# Attrition check: regression results
if (sum(q15_attrition$attrition_prop > 0) > 0) {
  regression_output(
    formula = "Q15_missing ~ q15_treat_clean", shown = rep(TRUE, nrow(d)),
                    survey_weights = d$survey_weights,
    variable_names = c("(Intercept)", "10 years", "20 years", "50 years"),
    caption = "Survey experiment attrition check: future time frame"
  )
}
```

      
```{r attritiontimeline3a, echo=FALSE, warning=FALSE, cache=TRUE}
d$q15_treat_10y <- d$q15_treat_clean == "10 years"
d$q15_treat_20y <- d$q15_treat_clean == "20 years"
d$q15_treat_50y <- d$q15_treat_clean == "50 years"
        
regression_output(formula = "Q15_clean ~ q15_treat_clean", shown = rep(TRUE, nrow(d)),
                          survey_weights = d$survey_weights,
                          variable_names = c("(Intercept)", "10 years", "20 years", "50 years"),
                          caption = "Survey experiment results: future time frame")  
```

```{r attritiontimeline3b, echo=FALSE, warning=FALSE, cache=TRUE}
regression_output(formula = "Q15_clean ~ q15_treat_10y + q15_treat_20y + q15_treat_50y + scale(Q15_missing) + q15_treat_10y:scale(Q15_missing) + q15_treat_20y:scale(Q15_missing) +q15_treat_50y:scale(Q15_missing)", shown = rep(TRUE, nrow(d)),
                          survey_weights = d$survey_weights,
                          variable_names = c("(Intercept)", "10 years", "20 years", "50 years",
                                             "DK/missing", "10 years x DK/missing",
                                             "20 years x DK/missing", "50 years x DK/missing"),
                          caption = "Survey experiment results: future time frame (controlling for DK/missing responses)", kable_output = c(1:4, 9))  
```
      
```{r attritiontimeline4, echo=FALSE, fig.height=7, fig.keep='all', fig.width=7, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf'}
md_Q15 <- regression_output(formula = "Q15_clean ~ q15_treat_10y + q15_treat_20y + q15_treat_50y + scale(Q15_missing) + q15_treat_10y:scale(Q15_missing) + q15_treat_20y:scale(Q15_missing) +q15_treat_50y:scale(Q15_missing)", shown = rep(TRUE, nrow(d)),
                    survey_weights = d$survey_weights,
                  variable_names = c("(Intercept)", "10 years", "20 years", "50 years",
                                     "DK/missing", "10 years x DK/missing",
                                     "20 years x DK/missing", "50 years x DK/missing"),
                  caption = "Survey experiment results: future time frame", 
                  output_model = TRUE) 

y10_20 <- linearHypothesis(md_Q15, c("q15_treat_10yTRUE = q15_treat_20yTRUE"), 
                        white.adjust = TRUE)
y10_50 <- linearHypothesis(md_Q15, c("q15_treat_10yTRUE = q15_treat_50yTRUE"), 
                 white.adjust = TRUE)
y20_50 <- linearHypothesis(md_Q15, c("q15_treat_20yTRUE = q15_treat_50yTRUE"), 
                           white.adjust = TRUE)

coef_equ_func <- function(dataset, test_text) {
  data.frame(test = test_text, f_stat = paste0("\\textit{F}(",
                                                         dataset$Df[2], ", ", 
                                                         dataset$Res.Df[2], ") = ",
                                                         roundfunc(dataset$F[2])),
           p_value = roundfunc(dataset$`Pr(>F)`[2]))  
}

coef_equ_timeline <- rbind(coef_equ_func(y10_20, "10 years = 20 years"),
                  coef_equ_func(y10_50, "10 years = 50 years"),
                  coef_equ_func(y20_50, "20 years = 50 years"))
kable(coef_equ_timeline, caption = "Testing coefficients for time frames are equivalent",
      row.names = FALSE, col.names = c("Tests", "\\textit{F}-statistic", "\\textit{p}-value"), 
      format = "pandoc")
```

\FloatBarrier 

## High-level machine intelligence: forecasting timeline {#apphlmi}

Figure \@ref(fig:hlmitimelinetabledemo1) displays the mean predicted the likelihood of high-level machine intelligence for each year by demographic subgroup. Figure \@ref(fig:hlmitimelinetabledemo2) displays the median predicted probability of high-level machine intelligence for each year by demographic subgroup. 

```{r hlmitimelinetabledemo1, echo=FALSE, fig.height=9, fig.keep='all',  warning=FALSE, cache=TRUE, results=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Mean predicted likelihood of high-level machine intelligence for each year by demographic subgroup"}

demo_hlmi_loop_res$demo_value <- factor(demo_hlmi_loop_res$demo_value,
                                        levels = rev(unique(demo_hlmi_loop_res$demo_value)))

ggplot(demo_hlmi_loop_res, aes(x = demo_value, y = Mean/100, shape = factor(hlmi_year),
                                color = factor(hlmi_year))) +
  geom_point() + coord_flip() + theme_bw() + xlab("Demographic subgroups") + 
  scale_y_continuous(labels = scales::percent, name = "Mean predicted likelihood", 
                     limits = c(0.425, 1)) +
  scale_shape_discrete(name = "Years") + scale_color_discrete(name = "Years") +
  theme(legend.position = "bottom")
```

```{r hlmitimelinetabledemo2, echo=FALSE, fig.height=9, fig.keep='all', warning=FALSE, cache=TRUE, results=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Median predicted likelihood of high-level machine intelligence for each year by demographic subgroup"}

ggplot(demo_hlmi_loop_res, aes(x = demo_value, y = Median/100, shape = factor(hlmi_year),
                                color = factor(hlmi_year))) +
  geom_point() + coord_flip() + theme_bw() +
  scale_y_continuous(labels = scales::percent, name = "Median predicted likelihood", 
                     limits = c(0.425, 1)) + xlab("Demographic subgroups") + 
  scale_shape_discrete(name = "Years") + scale_color_discrete(name = "Years") +
  theme(legend.position = "bottom")

```

## Support for developing high-level machine intelligence {#appsupporthlmi}

We examined the correlation between support for developing AI and support for developing high-level machine intelligence using a 2D bin count graph. The overall Pearson correlation coefficient is 0.61, according to Figure \@ref(fig:supporthlmicorr).

The mean level of support for developing high-level machine intelligence, compared with the mean level of support for developing AI, is 0.24 points (MOE = +/- 0.04) lower on a five-point scale (two-sided $p$-value $<0.001$), as shown in Table \@ref(tab:diffsupportaihlmi).

Table \@ref(tab:demosupporthlmiregtab1) displays the regression results used to produce Figure \@ref(fig:demosupporthlmi2).

To identify subgroups that have diverging attitudes toward high-level machine intelligence versus AI, we performed multiple regression using both the demographic subgroups variables _and_ respondents' support for developing AI as predictors. The support for developing high-level machine intelligence outcome variable was standardized such that it has mean 0 and unit variance. The results are shown in Table \@ref(tab:demosupporthlmiregtab2).

After controlling for one's support for developing AI, significant predictors correlated with support for developing high level machine intelligence, including:

- Being a member of the Silent Generation (versus being a Millennial/post-Millennial) 
- Having CS or programming experience (versus not having such experience)
- Having a high school degree or less (versus having at least a four-year college degree)

```{r supporthlmicorr, echo=FALSE, fig.height=5, results='hide', fig.keep='all',  warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Correlation between support for developing AI and support for developing high-level machine intelligence"}

# Make a new variable
d$Q5_corr <- d$Q5_clean
d$Q5_corr[!d$Q5_corr %in% c(-2:2)] <- 3

d$Q17_corr <- d$Q17_clean
d$Q17_corr[!d$Q17_corr %in% c(-2:2)] <- 3

# Convert to factor
labels_support <- rev(c("DK", "Strongly\nsupport", "Somewhat\nsupport",
                                "Neither support\nnor oppose", "Somewhat\noppose",
                                "Strongly\noppose"))
d$Q5_corr <- factor(d$Q5_corr, levels = c(-2:3), labels = labels_support)
d$Q17_corr <- factor(d$Q17_corr, levels = c(-2:3), labels = labels_support)

# Summarize the data
q5_17 <- d %>% group_by(Q5_corr, Q17_corr) %>%
  dplyr::summarise(prop = n()/nrow(d),
                   percent = roundfunc(n()/nrow(d)*100, 1))
# Correlation
corr_support <- wCorr::weightedCorr(x = d$Q5_clean, y = d$Q17_clean, 
                    weights = d$survey_weights, method = "pearson")
# Make the graph
ggplot() +
  stat_summary_2d(data = q5_17, aes(x = Q5_corr, 
                                    y = Q17_corr, z = prop)) + 
  geom_shadowtext(data = q5_17, aes(x = Q5_corr, 
                                    y = Q17_corr, label = percent),
                  color = "black",
                  bg.colour = "white", bg.r = 0.05) +
  theme_bw() +
  geom_smooth(data = d, aes(x = Q5_clean+3, y = Q17_clean+3, weight = survey_weights), 
              method = "lm", 
                           se = FALSE, linetype = 2, color = "red") +
  xlab("Support for developing AI") + 
  ylab("Support for developing high-level\nmachine intelligence") + 
  coord_equal() + 
  labs(title = paste0("Pearson's r = ", roundfunc(corr_support)), 
       caption = "Source: Center for the Governance of AI") + 
  scale_fill_gradient(name = "Percentage of respondents", labels = scales::percent,
                      low = "#f7fbff", high = "#08519c", limits = c(0, 14)/100,
                      breaks = c(0, 2, 4, 6, 8, 10, 12, 14)/100) +
  theme(legend.position = "bottom", 
        axis.text.x = element_text(angle = 270, hjust = 0)) +
  guides(fill = guide_colourbar(title.position="top", title.hjust = 0.5, 
                                barwidth = 14))
```


```{r diffsupportaihlmi, echo=FALSE, fig.height=5, fig.keep='all',  warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}

# Difference between support 
df_support <- data.frame(outcome = c(d$Q5_clean, d$Q17_clean), 
                        outcome_missing = c(d$Q5_missing, d$Q17_missing),
                        id = rep(d$caseid, 2),
                        type = c(rep("AI", 2000), rep("High-level machine intelligence", 2000)),
                        weights = rep(d$survey_weights, 2))
md_dff <- lm(outcome ~ type + scale(outcome_missing) +
               type:scale(outcome_missing), 
             data = df_support, weights = df_support$weights)
md_dff_s <- coef_test(md_dff, vcov = "CR2", cluster = df_support$id)
md_dff_s$stars <- ""
md_dff_s$stars[md_dff_s$p_Satt < 0.05] <- "*"
md_dff_s$stars[md_dff_s$p_Satt < 0.01] <- "**"
md_dff_s$stars[md_dff_s$p_Satt < 0.001] <- "***"

md_diff_out <- rbind(data.frame(vars = c("(Intercept)", "High-level machine intelligence"),
                         coef = paste0(roundfunc(md_dff_s$beta[1:2]), " (",
                                       roundfunc(md_dff_s$SE[1:2]), ")",
                                       md_dff_s$stars[1:2])),
                    data.frame(vars = "\\textit{N} = 2000",
                               coef = "")) 

# Output the table
kable(md_diff_out,
      format = "pandoc", row.names = FALSE, col.names = c("Variables", "Coefficients (SEs)"),
        caption = "Difference between support for developing AI and support for developing high-level machine intelligence")

```

```{r demosupporthlmiregtab1, echo=FALSE, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}

kable(x = dev_md_support_hlmi_k, row.names = FALSE, col.names = c("Variables", "Coefficients (SEs)"),
      format = "pandoc", caption = "Predicting support for developing high-level machine intelligence using demographic characteristics: results from a multiple linear regression that includes all demographic variables; outcome standardized to have mean 0 and unit variance")

```

```{r demosupporthlmiregtab2, echo=FALSE, warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7}
# Generate the data for the graph
dev_md_hlmi2_o <- lm_robust(formula = as.formula("Q17_clean ~ demo_age + demo_gender + 
               demo_white + demo_educ + demo_employ + 
    demo_pid3 + demo_income + demo_rel + demo_bornagain + demo_cs + 
    demo_prog + Q5_clean"), data = d, weights = d$survey_weights)

dev_md_hlmi2 <- data.frame(variables = names(dev_md_hlmi2_o$coefficients),
                     num = dev_md_hlmi2_o$coefficients,
                     se = dev_md_hlmi2_o$std.error,
                     p = dev_md_hlmi2_o$p.value)
dev_md_hlmi2$variables <- gsub(pattern = paste0(demo_var, collapse = "|"), replacement = "",
                         x = dev_md_hlmi2$variables)
dev_md_hlmi2$variables[dev_md_hlmi2$variables == "Q5_clean"] <- 
  "Support for developing AI"

# # Make the stars
dev_md_hlmi2$stars <- ""
dev_md_hlmi2$stars[dev_md_hlmi2$p < 0.05] <- "*"
dev_md_hlmi2$stars[dev_md_hlmi2$p < 0.01] <- "**"
dev_md_hlmi2$stars[dev_md_hlmi2$p < 0.001] <- "***"

# Make the text
dev_md_hlmi2$new_text <- paste0(roundfunc(dev_md_hlmi2$num), 
                      " (", roundfunc(dev_md_hlmi2$se), 
                      ")", dev_md_hlmi2$stars)


# add in the N
dev_md_hlmi2_k <- rbind(data.frame(variables = dev_md_hlmi2$variables, coef = dev_md_hlmi2$new_text),
                  data.frame(variables = "\\textit{N} = 2000",
                             coef = paste0("\\textit{F}(", round(dev_md_hlmi2_o$fstatistic[2]), ",",
                           round(dev_md_hlmi2_o$fstatistic[3]), ") = ",
                           roundfunc(dev_md_hlmi2_o$fstatistic[1]), "; \\textit{p}-value: <0.001")))

kable(x = dev_md_hlmi2_k, row.names = FALSE, col.names = c("Variables", "Coefficients (SEs)"), 
      format = "pandoc", caption = "Predicting support for developing high-level machine intelligence using demographic characteristics: results from a multiple linear regression that includes all demographic variables and respondents' support for developing AI; outcome standardized to have mean 0 and unit variance")

```

\newpage

## Expected outcome of high-level machine intelligence {#appexpectedoutcome}

We examined the correlation between respondents' expected outcome of high-level machine intelligence and support for developing high-level machine intelligence using a 2D bin count graph. The overall Pearson correlation coefficient is 0.69, as seen in Figure \@ref(fig:corroutcomehlmi).

```{r corroutcomehlmi, echo=FALSE, fig.height=5, results='hide', fig.keep='all', warning=FALSE, cache=TRUE, dpi = 300, dev = 'pdf', fig.width=7, fig.cap="Correlation between expected outcome and support for developing high-level machine intelligence"}

# Make a new variable
d$Q18_corr <- d$Q18_clean
d$Q18_corr[!d$Q18_corr %in% c(-2:2)] <- 3

d$Q17_corr <- d$Q17_clean
d$Q17_corr[!d$Q17_corr %in% c(-2:2)] <- 3

# Convert to factor
d$Q18_corr <- factor(d$Q18_corr, levels = -2:3, labels = 
                       c("Extremely\nbad", "On balance\nbad", "More or\nless netural",
                         "On balance\ngood", "Extremely\ngood", "DK"))
d$Q17_corr <- factor(d$Q17_corr, levels = c(-2:3), labels = labels_support)

# Summarize the data
q18_17 <- d %>% group_by(Q17_corr, Q18_corr) %>%
  dplyr::summarise(prop = n()/nrow(d),
                   percent = roundfunc(n()/nrow(d)*100, 1))

corr_outcome <- wCorr::weightedCorr(x = d$Q18_clean, y = d$Q17_clean,
                  weights = d$survey_weights, method = "pearson")

# Make the graph
ggplot() +
  stat_summary_2d(data = q18_17, aes(x = Q18_corr, 
                                    y = Q17_corr, z = prop)) + 
  geom_shadowtext(data = q18_17, aes(x = Q18_corr, 
                                    y = Q17_corr, label = percent),
                  color = "black",
                  bg.colour = "white", bg.r = 0.05) +
  theme_bw() +
  geom_smooth(data = d, aes(x = Q18_clean+3, y = Q17_clean+3), method = "lm", 
                           se = FALSE, linetype = 2, color = "red") +
  xlab("Expected outcome of high-level machine intelligence") + 
  ylab("Support for developing high-level\nmachine intelligence") + 
  coord_equal() + 
  labs(title = paste0("Pearson's r = ", roundfunc(corr_outcome, 2)),
       caption = "Source: Center for the Governance of AI") + 
  scale_fill_gradient(name = "Percentage of respondents", 
                      low = "#f7fbff", high = "#08519c",
                      breaks = c(0, 2, 4, 6, 8, 10, 12, 14)/100,
                      labels = scales::percent) +
  theme(legend.position = "bottom", 
        axis.text.x = element_text(angle = 270, hjust = 0)) +
  guides(fill = guide_colourbar(title.position="top", title.hjust = 0.5, 
                                barwidth = 15))

```

\newpage

# About us {-}

## About the Center for the Governance of AI {-}

The Center for the Governance of AI, housed at the Future of Humanity Institute, University of Oxford, strives to help humanity capture the benefits and mitigate the risks of artificial intelligence. Our focus is on the political challenges arising from transformative AI: advanced AI systems whose long-term impacts may be as profound as the industrial revolution. The Center seeks to guide the development of AI for the common good by conducting research on important and neglected issues of AI governance, and advising decision makers on this research through policy engagement.

The Center produces research which is foundational to the field of AI governance, for example mapping crucial considerations to direct the [research agenda](https://www.fhi.ox.ac.uk/wp-content/uploads/AI-Governance_-A-Research-Agenda.pdf), or identifying distinctive features of the transition to transformative AI and corresponding [policy considerations](https://www.fhi.ox.ac.uk/wp-content/uploads/Policy-Desiderata-in-the-Development-of-Machine-Superintelligence.pdf). Our research also addresses more immediate policy issues, such as [malicious use](https://maliciousaireport.com/) and [China’s AI strategy](https://www.fhi.ox.ac.uk/wp-content/uploads/Deciphering_Chinas_AI-Dream.pdf). Our work takes a cross-disciplinary approach, looking at transformative AI through the lenses of e.g. international security, the history of technology development, law and public opinion.

In addition to research, the Center for the Governance of AI is active in international policy circles, and actively advises governments and industry leaders on AI strategy. The Center for the Governance of AI researchers has spoken at the NIPS and AAAI/ACM conferences, and at events involving the German Federal Foreign Office, the European Commission, the European Parliament, the UK House of Lords, the U.S. Congress, and others. 

The Center's papers and reports are available at [https://www.governance.ai](https://www.governance.ai).  

## About the Future of Humanity Institute {-}

The Future of Humanity Institute, University of Oxford, is a multidisciplinary research institute at the University of Oxford. Academics at FHI bring the tools of mathematics, philosophy and social sciences to bear on big-picture questions about humanity and its prospects. The Institute is led by Founding Director Professor Nick Bostrom. Humanity has the potential for a long and flourishing future. Our mission is to shed light on crucial considerations that might shape that future. More information at http://www.fhi.ox.ac.uk/.

\newpage

# References
